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Introduction 



In 1992, the CCCC Committee on Assessment was charged with the 
responsibility of inquiring into the state of postsecondary writing assessment in the 
United States. The charge to the Committee followed a period of rapid change in 
the teaching of writing and a growing concern held by many in the field that our 
assessment methods were not keeping pace with that change. In recent decades, 
research and practice in writing have led to a view of writing as a meaning-making 
activity requiring the orchestration of diverse skills and strategies in the completion 
of tasks that vary with purpose, audience and context (Langer & Applebee, 1987; 
Odell, 1981; Camp, 1993; Hairston, 1982; White, 1985). Simultaneously, there has 
been a growing recognition of the inadequacies of many of our traditional methods 
; ^ assessing writing and a growing dissatisfaction with the gap between assessment 
practices and teachers' views of effective pedagogy. Critics point to the mismatch 
between multiple-choice methods and theoretical conceptions of writing as a 
complex, multi-faceted cognitive activity. They warn of the potentially iiividious 
influence of such tests on curriculum, pointing out that such tests encourage 
fragmentation of the curriculum (Haertel & Calfee, 1983), that they focus on the less 
significant components of writing (spelling, usage, punctuation) (Brown, 1978; 
Greenberg, 1992) and that they emphasize passive choice as opposed to active 
construction (Johnston, 1985). 

Multiple-choice assessment has not been the only target for criticism. While 
there may be a growing consensus in the field that the "best way to assess students' 
writing skills is through writing or "direct" assessment," (Greenberg, 1992) critics 
point to the inadequacies of single-sample testing, expressing their skepticism about 
the idea that a single sample can adequately represent the variety of discourse 
modes, purposes and audiences which writing entails (Emig, 1982; Lucas, 1988), and 
their concern that single-topic, one domain testing may encourage a blurring of 
distinctions in instruction among text features that are appropriate for different 
kinds of writing tasks. So widespread is the concern about single sample testing that 
in November, 1992, the National Council of Teachers of English passed a Sense of 
the House Motion (No. 3) expressing opposition to single sample testing: "Be it 
resolved that the Council oppose the practice of claiming to measure a student's 
overall ability at writing by means of a single score on a single piece of writing 
produced at one sitting, and be it further resolved that the Council work to 
eliminate this practice." 

Concern is also growing about the potentially negative impact of assessment 
on faculty and students (Beard, et al., 1985). Critics point to the improper uses to 
which tests may be put and to the potentially negative impact on students of 
assessment results based on erroneous judgments based on inadequate measures. 
Critics also express concern about the potentially negative impact of assessment on 
faculty, in particular, the loss of authority and professional prerogative that occurs 
when assessment is mandated by external authorities (Madaus, 1985; Lucas, 1988; 
Pearson & Valencia, 1987; Murphy & Grant, in press) and they call for the 
involvement of faculty in the assessment process (Cuba & Lincoln, 1989). 
Greenberg, for example, points out the role which faculty play in establishing 
validity: "Faculty who share ideas and work together to develop an essay test often 
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shape an exam that is grounded in their theories, curricula, and classroom practices" 
(1992, 15). 

In higher education, writing assessment has come to play an increasingly 
important role as a source of information for making decisions about the admission 
of students, their placement into particular courses, and their proficiency. Reflecting 
widespread professional concern over the potentially negative effects of testing 
programs on curriculum, faculty, and students, the CCCC and its parent 
organization the NCTE have drafted a number of resolutions about assessment 
practices during the last two decades. Beginning in 1971, the NCTE passed a 
resolution urging school districts, colleges and state agencies to re-examine 
standardized tests "in order to determine the appropriateness of their content to 
actual instructional goals" and to "consider carefully means other than standardized 
tests, including student self-evaluation, of assessing the language arts skills of 
students." Subsequent resolutions urged faculty and administrators to develop 
alternatives to standardized testing (1976), to avoid tests that focused on subskills 
(1977), and to reject mandated assessments as criteria for promotion and/or 
graduation of students (1977). In reaction to continued concern that "multiple- 
choice and short answer tests yield httle diagnostic information of use to teachers 
and continue to disrupt the curriculum and distract teachers from working to 
develop students' higher order thinking skills/ a 1985 NCTE resolution called for 
the Council to seek ways to "empower English teachers," and "to seek to develop 
new and alternative models of testing and assessment." In 1978, CCCC passed a 
Resolution on Testing that recommended guidelines for conducting postsecondary 
writing assessments. These guidelines, addressing the question of authority over 
the assessment process, recommended that "Responsibility for giving credit, 
exemption, or accreditation" should rest, "not with local administrators or state 
officials, but with the composition faculty in each institution," and further, that tests 
of writing be "administered under the primary control and supervision of 
representatives of the composition faculty." More recently, the NCTE passed the 
previously mentioned Sense of the House Motion opposing single score, single 
sample, single sitting testing (1991). 

In addition to taking positions on assessment issues, both CCCC and NCTE 
have estabUshed committees to advise their memberships about assessment issues. 
In 1980, the NCTE established its Committee on Testing and Evaluation, which in 
1985 was made a Standing Comnaittee. In 1991 NCTE and IRA joined forces to 
establish the joint NCTE/IRA Task Force on Literacy Assessment. In 1978, the CCCC 
established the Committee on Assessment (then called the Committee on Testing). 
One role played by these committees has been to gather information about 
assessment practices. In the case of CCCC committees, the focus has been on 
assessment practices at postsecondary institutions. In 1979 and again in 1981 
(Purnell, 1982), the CCCC Task Force on Testing conducted a survey on proficiency 
testing. While members of the Task Force acknowledged the "potential benefits for 
our profession" of testing, they also warned that "it surely can present grave threats 
tC) the equity and integrity of the educational process" and they recommended "a 
continuing study of both the products and process of testing, its uses and abuses." In 
1985, the CCCC Committee on Assessment conducted a survey of writing 
assessment practices at postsecondary institutions and prepared a report for the 



CCCC Executive Committee. In 1992, as noted above, the CCCC Committee on 
Assessment was again authorized to conduct a survey to describe the current writing 
assessment practices at postsecondary institutions of higher education in the United 
States. 



The Survey 

Members of the Committee designed and piloted a thirty-item questionnaire 
to collect information about many of the areas of concern described above. 
Specifically, the survey asked questions about the following practices in writing 
assessment in postsecondary institutions: 1) the kinds of decisions which may be 
based, in whole or in part, on results of an assessment of writing, 2) the methods 
employed to assess writing, 3) the roles of participants in the assessment process, 4) 
the consequences for students, 5) the distribution of authority over the assessment 
process, and 6) opinions of respondents about the impact of assessment on writing 
instruction and their satisfaction or dissatisfaction with the methods employed at 
their campuses. Demographic information about respondents was also collected. 

Based on the demographic information provided by the U. S. Department of 
Education, a stratified random sample was drawn from^ the total population of 3,504 
institutions of higher education which represented public and private institutions, 
and institutions by size of enrollments (small, medium, large, very large). With 
financial support provided by the NCTE, a total of 842 surveys were mailed to the 
institutions sampled. Letters accompanying the surveys were addressed either to 
the directors of writing programs at each institution or to the chairs of English 
departments. A second mailing 5 weeks following the first was sent to institutions 
that had not responded. Follow up calls were made to 40 of the smaller colleges to 
obtain a more representative sample of them. 

A total of 534 completed surveys were returned, representing 15 percent of the 
population and a 63 percent response rate. As a whole, the sample is substantially 
larger than the minirnum 10 percent required for descriptive research and allows 
generalization to the population (Gay, 1987, 114-115). For a sample of 500, Lauer and 
Asher (1988) indicate 95% confidence interval limits of plus and minus 4.4%, (p.58). 
With a correction factor of .92 for a 15% sample, (p.61) the confidence limits for this 
sample narrow to plus and minus 4.05%. Moreover, the relatively high rate of 
return suggests that in general, the sample is unbiased. Analysis of the responses 
indicated that campuses responded in relatively close proportion to the strata on 
which the sample was based (size and public/private status) (see Table 1). 

Table 1 

Rate of Return by Type of Institution by Institution Size 



Small 
Medium 
Large 
Ex- Large 



Public 




Private 


7r 


19/153 


(.12) 


97/1174 


(.08) 


126/712 


(.18) 


130/666 


(.20) 


108/571 


(.1^) 


27/113 


(.24) 


27/132 


(.20) 


0/9 


(.0) 
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In only two cases did subsamples fall below 10% of the population on which 
they were based. The subsample for small, private institutions (^7) is only 8% of the 
population of small, private institutions in the United States (1174). In addition, the 
sample as a whole does not include responses from any of the 9 extra-large, private 
institutions in the United States. However, each of the other subsamples is more 
than 10% of the population on which it is based. 

The survey data were entered on a mainframe computer, checked for 
reliability, and summarized in terms of frequencies of responses to the different 
questions and other analyses, such as cross-tabulations, by Optimum Data Systems. 
Additional cross-tabulations were completed by an employee oi the CRESS Center at 
U.C. Davis. These data were then analyzed and interpreted by the co-chair and a 
former member of the Committee on Assessment. 

The results of this survey were an.vvzed from three perspectives — responses 
provided by (a) all respondents regardless of category of institution or size, 

(b) respondents categorized as representing either public or private institutions, and 

(c) respondents categorized by size of institutions (enrollments). The latter analyses 
were undertaken to determine the extent to which practices differed, depending on 
the kinds of support received or the numbers of students for which decisions were 
being made. While the results for all respondents are presented in detail here, due 
to space limitations, only selected results of the other analyses are mentioned. A 
more complete report of the survey results will be submitted to the Executive 
Committee and will be made available on ERIC. The following discussions begin 
with the d^^mographics for, and results supplied by, all respondents. 



Demographics of the Survey Respondents 

The individuals who responded to the survey consisted of 166 faculty 
members, 95 writing program administrators, 216 department chairpersons, 31 
deans, and six non-instructional staff. According to responses to a question about 
the nature of the institutions, the campuses they represented were 174 two-year, 129 
four-year, 208 four-year with advanced degree programs, and 5 graduate institutions 
(18 of the institutions failed to respond to the question asking them to classify their 
campus). Based on information supplied by the U.S. Department of Education, 245 
of the institutions were classified as private and 264 were public. The institutions 
were further classified on the basis of their enrollments as Extra Large (20,000+), 
Large (5,000-19,999), Medium (1,000 - 4,999) and Small (1-999). Of the institutions 
which responded, 27 were Extra Large, 135 were Large, 256 were Medium, and 116 
were Small. 



SECTION I: OVERALL RESULTS 

Assessment Methods and Decisions Based on Assessment Results 

To gather information about the extent to which writing assessment plays a 
role in critical decision-making processes in postsecondary institutions, questions in 



the first section of the survey asked whether or not the following critical decisions 
about students were based on any type of assessment of writing skills: 

(a) admission to campus, e.g. essay questions on the campus's appUcation 
form, SAT scores, etc., 

(b) exemption from regular freshman English, e.g. AP tests, CLEP tests, 
campus developed exams, etc., 

(c) placement in freshman En<^lish or writing courses, e.g. pxams for students 
new to the writing curriculum, 

(d) proficiency for entry into a next level of coursezvork, e.g. for students 
moving from the highest level basic writing course to the first semester of 
regular freshman English, or for students moving from freshman- 
sophomore standing to junior-senior standing, and 

(e) condition of graduation (exit) e.g., proficiency which must be 
demonstrated prior to graduation. 

For the sake of brevity, the terminology describing the decisions based on 
assessment results, or purposes for using writing assessment, have been shortened 
in the text to "admission" for admission to campus, "exemption" for exemption 
from regular freshman English, "placement" for placement in freshman English or 
writing courses, "proficiency" for proficiency for entry into a next level of 
coursework, and "graduation" for condition of graduation. 

Which Decisions are Based on the Assessment of Writing Skills ? 

Of the 534 respondents, the numbers and percentages of those who indicated 
whether or not they based certain decisions, in whole or in part, on any type of 
assessment of writing skills were as follows: 

Table 2 

Decisions Based on Any Type of Assessment of Writing Skills 

(N=:534) 



No 

Yes No Response 





# 


% 


# 


7c 


# 


7f 


Adnnission 


232 


43 


269 


50 


33 


6 


Exemption 


381 


71 


116 


22 


37 


7 


Placement 


421 


79 


82 


15 


31 


6 


Proficiency 


304 


57 


186 


35 


44 


8 


Graduation 


172 


32 


320 


60 


42 


8 



For those who indicated that decisions on their campuses are based, in whole 
or in part, on any type of assessment of writing skills, a majority of the total number 
of respondents use some form of writing assessment for placement (421, 79%) and 
exemption C 81, 71%) decisions. A considerable number (304, 57%) also use writing 
assessment to make decisions regarding exemption. Of interest is the large number 
of institutions that use writing assessment as a condition of graduation (172, 327o). 
However, it should be noted that because the question asked about "any" type of 
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assessment of writing, the term "assessment" could be interpreted broadly to mean 

writing requirements which are integrated into required courses set by the WM 

institution as conditions for graduation (as well as more formal assessments) as 

forms of writing assessment. The results indicate that, overall, most of the decision- 

making based on writing assessment revolves around the freshman English ce-)urse. |H 

What Methods are Used for the Different Assessment Purposes ? 

Question 1 of the survey asked about any type of writing assessment, 
including untimed writing samples such as the essays which are submitted with 

applications for admission. Question 2, on the other hand, excluded untimed |H 
writing samples, focusing instead on three particular assessment methods: Multiple- 

choice exams, timed writing samples, and portfolios. Responses to the two |H 

questions thus differed substantially. In addition, however, several respondents H 

who had indicated that they used writing assessment for certain purposes in m 

question 1 did not respond to question 2. Some respondents may have found this H 

question complex and thus difficult to answer. An additional cross-check of H 

respondents' answers revealed that a few individuals who did not respond to H| 

question 2 later answered specific questions about a particular method. The lesults H 

in this section, then, must be interpreted cautiously. H 



Table 3 

Approximate Numbers of Institutions Using Multiple Choice, 
Timed Writing Samples and/or Portfolios for Decision Making 





Use Writing 


Multipl 


e 


Timed 






Assessment 


Choice Exams 


Writing 


Samples 


Portfolios 




# 


# 


% 


# 


7r 


# 


Adirdsbion 


232 


1% 


84 


102 


44 


32 14 


Exemption 


381 


259 


68 


271 


71 


46 12 


Placement 


421 


316 


75 


323 


77 


22 5 


Proficiency 


3()4 


129 


42 


241 


79 


110 36 


Graduation 


172 


78 


45 


126 


73 


45 26 



[Percentages are based on the total number of institutions reporting that they use 
writing assessment for each particular decision making purpose. The numbers 
reported for each assessment method include all institutions which report using 
the method, whether alone or in combination with another assessment method.] 
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Approximate numbers of institutions using multiple choice exams, timed 
writing samples and/or portfolios alone, or in combination with another method 
are reported in Table 3. The estimates for multiple choice and timed writing 
samples are based on the numbers of institutions who responded to specific 
questions regarding the development and implementation of those particular 
assessment methods. The estimates for portfolios are based upon the number of 
institutions reporting their use for each of the purposes in response to question 2. 
Because respondents at a number of the institutions which use writing assessment 
for particular decisions failed to respond to question 2, the numbers reported for 
portfolios may be an underestimate of their actual use. 
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Responses indicate that of the institutions using writing assessment to make 
decisions about admission, approximately 84% use multiple choice exams, 
approximately 447o use timed writing samples, and approximately 14% use 
portfolios. For making decisions about exemption, approximately 68% use multiple 
choice, while 71% use timed writing samples, and 12% use portfolios. For making 
decisions about placement, approximately 75% use multiple-choice, 77% timed 
writing samples, and 5%, portfolios. For making decisions about proficiency, 
approximately 42% use multiple-choice exams, 79%, timed writing samples and 36% 
portfolios. For making decisions about graduation, approximately 78% use 
multiple-choice exams, 73% timed writing samples, and 26% portfolios. 

Practices Associated with Timed Writing Samples 

Several of the survey questions focused on the assessment process, asking 
who develops, scores, and/or makes decisions about the results of particular exams, 
how particular exams are implemented and scored, and what kinds of feedback 
students receive. In an effort to create as brief a survey as possible, questions in this 
section of the survey dealt only with multiple-choice and timed writing 
assessments. Specific questions about the development and implementation of . 
portfoUo assessment were not posed because at the time the survey was developed, 
it was expected that few institutions were using this method to assess writing. 

How Many Samples are Collected in Timed Writing Assessments ? 

hi response to concern in the profession about the extent to which procedures 
for conducting writing samples relate to current theoretical conceptions of writing a 
complex, multi-faceted cognitive activity and the extent to which assessment 
methods reflect teachers' views of effective pedagogy, the Committee included 
several questions in the survey about specific assessment practices. In the case of 
assessments in which timed writing samples were collected, these included 
questions about the number of samples collected, the time allowed for writing, 
whether prewriting and revising activities were encouraged, and whether students 
received feedback about their performance. Of particular interest to the Committee 
was the extent to which institutions using writing assessment in decision-making 
relied on single sample testing. 



Table 4 

Numbers of Writing Samples Collected 

One Two Three Four or more 





Sampl 


e 


Sampl 


es 


Sampl 


es 


Samples 




# 


% 


# 


% 


# 


% 


# % 


Admission 


45 


82 


5 


9 


0 


0 


5 9 


Exemption 


153 


79 


21 


11 


9 


5 


10 5 


Placement 


205 


85 


21 


9 


6 


2 


10 4 


Pro ficicncy 


103 


65 


20 


13 


9 


6 


26 16 


Graduation 


66 


83 


6 


8 


3 


4 


5 6 



[Percentages based on the number of institutions responding to each question] 

7 



ERIC J 



Of the institutions indicating that timed writing samples are collected to 
make decisions about admission, (45, 827o) rely on a single sample, about exemption, 
(153, 79%), about placement, (205, 85%), about proficiency, (103, 65%), and about 
graduation, (66, 83%). Thus, in spite of professional concern about the practice of 
making generalizations about overall writing ability from a single sample collected 
on a single occasion, this seems to be the most common practice in the direct 
assessment of writing in which timed writing samples are collected. 

How Much Time Is Allowed for Writmg in Timed Writing Assessments ? 

The amount of time students are allowed to complete writing samples is an 
issue of concern to professionals who want to see a closer match between assessment 
procedures and teachers' views of effective pedagogy. Assessments which allow brief 
amounts of time tap only first draft writing while effective pedagogy encourages 
students to plan and revise their writing. Further, there is some evidence that 
linguistic minority students are particularly disadvantaged by abbreviated time 
frames (Hilgers, 1992). To gather information about this issue, the survey asked 
respondents who used timed writing samples to make assessment decisions to 
indicate the amount of time allowed. The results are presented in Table 5. 

Of the institutions indicating they use timed writing samples in assessment to 
make decisions about admission, (22, 35%) allow 30 minutes or less, (27, 43%) allow 
31 to 60 minutes, (8, 13%) allow 61 to 90 minutes, while (6, 9%) allow more than 90 
minutes» Of those using timed writing samples to make decisions about exemption, 
(31, 14%) allow 30 minutes or less, { 101, 45%) allow 31 to 60 minutes, (45, 20%) allow 
61 to 90 minutes, while (48, 21%) allow more than 90 minutes. Of those using timed 
writing samples to make decisions about placement, (61, 19%) allow 30 minutes or 
less, (101, 56%) allow 31 to 60 minutes, (44, 14%) allow 61 to 90 minutes, while 
(33, 11%) allow more than 90 minutes. Of those using timed writing samples to 
make decisions about proficiency, (18,18%) allow 30 minutes or less, (93, 42%) allow 
31 to 60 minutes, (45, 20%) allow 61 to 90 minutes, while (66, 30%) allow more than 
90 minutes. 



Table 5 

Average Times AU-^wed To Complete a Writing Sample 





<30 




31-60 


61-90 


90+ 






# 


% 


# 




# 


% 


# 




Admission 


22 


35 


27 


43 


8 


13 


6 


10 


Exemption 


31 


14 


101 


45 


45 


20 


48 


21 


Placement 


61 


19 


175 


56 


44 


14 


33 


11 


Proficiency 


18 


H 


93 


42 


45 


20 


66 


30 


Graduation 


6 


6 


43 


40 


18 


17 


40 


37 



Of those using timed writing samples to make decisions about graduation, 
(6, 6%) allow 30 minutes or less, (43, 407o) allow 31 to 60 minutes, (18, 17%) allow 61 
to 90 minutes, while (40, 37%) allow more than 90 minutes. Thus, almost regardless 
of purpose, most institutions allow students an hour or less to complete a writing 
sample: 78%. for admission, 59% for exemption, and 75% for placement. Half of the 
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institutions using tin\ed writing samples (50%) allow an hour or less for proficiency 
exams, while slightly fewer (46%) allow an hour or less for writing samples 
collected to make decisions about graduation. Relatively more time seems to be 
allowed in assessments related to proficiency and graduation than in assessments 
related to admission, exemption and placement. 

Are Prewriting and/or Revising Activities Encouraged When Timed Writing 
Samples Are Collected? 

In order to gather information about the degree to which assessment 
procedures mirrored typical patterns in writing instruction, respondents were also 
asked to indicate whether or not they encourage prewriting and/or revising 
activities during timed writing tasks. Most institutions reported they did not 
encourage prewriting and/or revising activities during assessments conducted for 
purposes of admission (65, 77%). For exemption and placement, however, the 
numbers of institutions are divided nearly equally among those who indicate they 
encourage prewriting and/or revising activities (see Table 6). However, for 
proficiency and for graduation, the pattern seen for admission is reversed. Most 
respondents indicate they encourage prewriting and/or revising activities when 
timed writing tasks are used to assess proficiency for entry into the next level of 
coursework (173, 71%) or as a condition of graduation (79, 64%). It appears that 
many institutions acknowledge the value of these activities beyond the freshman 
level of education. Given the relatively limited amount of time allowed for writing 
in many of these assessments, however, there is some question as to the extent such 
activities may realistically be possible. 

Table 6 

Prewriting and/or Revising Activities in the Collection of Timed Writing Samples 





Yes 




No 






# 


% 


# 




Admission 


19 


23 


65 


77 


Exemption 


107 


45 


133 


55 


Placement 


169 


52 


154 


48 


Proficiency 


173 


71 


70 


29 


Graduation 


79 


64 


45 


36 



What Methods are Used to Scoie Timed Writing Samples ? 

The kinds of scoring methods used at the different institutions for the 
various purposes were also reported by the respondents. For purposes of uniformity 
in the use of terminology, analytic scoring was defined as scoring of general features 
of writing such as ideas, organization, usage, punctuation, etc. General impression 
scoring was defined as holistic scoring of the essay as a whole, while primary trait 
scoring was defined as scoring of the rhetorical features of a piece of writing which 
are relevant to the kind of discourse i^ is, e.g. narrative, memoir, argument, etc. 
Respondents were asked to check all that apply. Of those institutions which indicate 
that they use timed writing samples in making a decision, a single scoring 



method — general impression scoring — appears to be predominant for all purposes: 
(46, 57%) for admission, (118, 47%) for exemption, (176, 54%) for placement, 
(111, 46%) for proficiency, and (61, 50%) for graduation. Several institutions, 
however, report using analytic scoring, (19,23%) for admission, (57, 23%), for 
exemption (52, 16%) for placement (49, 21%) for proficiency and (19, 16%) for 
graduation. Several institutions use both analytic and general impression scoring 
for the scoring of timed writing samples for admission, exemption, proficiency, and 
graduation decisions (see Table 7). Primary trait scoring is used by only a few 
institutions (see Table 7). 
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Is Feedback Provided to Students about the Results of Timed Writing Assessments? 



An issue in assessment procedures is the question of whether students 
should receive feedback about their performance. For pragmatic reasons, (expense, 
time) some institutions may not provide feedback. However, some educators argue 
that assessment should contribute to learning and that feedback is thus a necessary 
part of the assessment process. To gather information relevant to this issue, two 
questions involving timed writing samples asked about opportunities for students 
to receive feedback about their performance and the kinds of feedback that students 
receive were included in the survey. 

Table 8 

Feedback to Students About Performance on Timed Writing Samples 



Opportunity for Students to Receive Feedback 





Yes, 




Yes, if 


No, no 




automatic 


reque 


sted 


feedback 




# 




# 


7r 


# 7c 


Admission 


31 


32 


24 


25 . 


42 43 


Exemption 


78 


30 


126 


48 


56 22 


Placement 


113 


34 


163 


49 


60 18 


Proficiency 


120 


49 


106 


44 


17 7 


Graduation 


44 


36 


72 


59 


6 5 



At the point of admission, most institutions (56, 57%) provide feedback, (31, 
32 %) automatically and (24, 25%) if requested. However, many institutions (42, 
43%) do not provide feedback at all. Beyond admission, most institutions provide 
feedback — most, automatically for proficiency (120, 49%; most, by request for 
exemption (126, 48%), placement (163, 48%), and graduation (72, 59%). For those 
institutions who use timed writing samples for condition of graduation, very few (6, 
5%) do not provide feedback about performance. 

Certain kinds of feedback may b:? more helpful to students than others. 
Scores, for example, may allow a student to assess the strength of his or her 
performance, but they will not provide information about areas of strength or 
weakness nor how to improve the performance. To explore this issue, information 
about the kinds of feedback provided was also collected (See Table 9). 
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Approximately half of the institutions which provide feedback about writing 
assessment for admission decisions (31, 50%) only provide scores, while half 
provide written comments, comments in conference, or some combination of the 
three kinds of feedback. Beyond admission, however, relatively more institutions 
provide comments in conference (81, 40%) about exemption assessment, (117, 417c) 
about placement assessment, (72, 33%) about proficiency, and (38, 34 7o) about 
assessment as a condition of graduation or some combination of written comments 
and/ or comments in conference in addition to scores. 

Participants in the Assessment Process 

As noted in the introduction, some professionals argue for the value of direct 
involvement of faculty in the assessment process. They point out that when faculty 
are involved in development, assessments are more likely to be grounded in the 
faculty's "theories, curricula, and classroom practices" (Greenberg, 1992, 15). It has 
also been argued that participation in the assessment process can serve as a vehicle 
for effective staff development, with the caution that "a good test can have as 
positive an effect on mediocre teaching as a bad test has negative effects on teaching" 
(Lucas, 1988, 2). To gather information about the extent of faculty involvement in 
the writing assessment process in postsecondary institutions, specific questions 
about faculty involvement were included in the survey. 

Who Develops Multiple Choice Exams ? 

Among the institutions who use multiple choice exams, most exams are 
developed by the staff of a commercial agency (such as the ACT, SAT, or CLEP) for 
the assessment of writing at the pre-freshman and freshman levels — for admission 
(176, 907o), exemption (198, 76%), and placement (223, 71%). This finding is not 
surprising, Sxace standardized tests are commonly used, particularly for admission 
and exemption (such as the AP English examination). Evidently many institutions 
also extend the appHcation of these scores to placement in freshman English or 
writing courses. Of those using multiple choice exams to assess writing proficiency 
for entry to the next level of coursework, the multiple choice exams are developed 
at most institutions (69, 53%) by their own faculty or staff, although many 
institutions (43, 33%) use commercially prepared exams. When used as a condition 
for graduation, many (36, 46%) multiple choice exams are developed by faculty of 
staff at their own college or univc-rsity, but many (24, 31%) also are developed by a 
commercial agency, and some (16, 20%), by a state agency or department (Table 10). 
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Table 10 

Participants in the Assessment Process: Multiple Choice Exams 

Who Develops Who Sets the Standards 









Faculty/ 


Staff 


at 






Faculty/ 


Diari <ii 




Staff of a 


staff at own 


^tate 


agency 


Administrator 


staff at own 


state agency 




commercial 


college 


or 




or 




at own college 


college or 


or 




^^gency 


universitv 


department 


or universitv 


universitv 


department 




M choice 


M choice 


M choice 


M choice 


M choice 


M choice 




# 


7. 


# 


7r 


# 






# 




# 


7r 


# 7 


Admission 


176 


93 


8 


4 


6 




3 


108 


61 


56 


32 


13 7 


Exemption 


198 


80 


39 


16 


11 




4 


46 


19 


175 


74 


15 6 


Placement 


223 


78 


45 


16 


19 




7 


35 


12 


234 


82 


17 6 


Proficiency 


43 


36 


69 


57 


9 




7 


9 


6 


128 


91 


4 3 


Graduation 


24 


32 


36 


47 


16 




21 


11 


13 


62 


73 
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Who Sets the Standards for Multiple Choice Exams? 

The individual who sets the standards (e.g., who decides which scores are 
acceptable) at institutions using multiple choice writing assessments for the purpose 
of admission is predominantly the administrator (108, 61%) at the college or 
university, with a relatively large number of institutions involving their faculty (56, 
32%) in this process. As would be expected, faculty are typically not active in the 
admissions process. For all other purposes, however, the facuUy at most 
institutions set the standards. For exemption decisions, respondents indicated that 
standards are set primarily by faculty (175, 74%) and some by administrators (46, 
19%); for placement, primarily by faculty (234, 827o); for proficiency, primarily by 
faculty (128, 91%); and for graduation, primarily by faculty (62, 73%) as well as by 
some administrators (11, 13%) and state agencies (12, 14%) (Table 10). 



Table 11 

Participants in the Assessment Process: Timed Writing Samples 



Who Develops 

Faculty/ 
Staff of a staff at own 
commercial college or 
a gency university 



Who Sets the Standards 



Staff at 
state agency 
or 

department 



Administrator 
at own college 
or university 



Faculty/ 
staff at own 
college or 
university 



Staff at 
state agency 
or 

department 



W samples W samples W samples W samples W samples W samples 





# 


7r 


# 


% 


# 


% 


# 




# 


% 


# 


7f 


Admission 


42 


43 


51 


^2 


5 


5 


43 


44 


53 


54 


2 




Exemption 


73 


28 


179 


b9 


6 


2 


20 


8 


231 


89 


• 10 


4 


Placement 


26 


8 


273 


88 


12 


4 


10 


3 


298 


93 


12 


4 


Proficiency 


8 


3 


228 


95 


3 


1 


6 


2 


237 


97 


2 


1 


Graduation 


9 


7 


101 


81 


15 


12 


9 


7 


102 


80 


16 


13 



15 



Who Develops the Topics for Timed Writing Samples? 



In contrast to the results for multiple choice exams, among the institutions 
that use timed writing samples for their assessment purposes, most of the 
assessments were developed by faculty or staff at their own colleges or universities. 
The one exception is for admission — nearly half the respondents (42, 43%) used 
topics for timed writing samples developed by a commercial agency and half (51, 
327g), by faculty or staff. These data suggest, as previously mentioned, that the 
commonly used standardized tests which include a writing sample provide scores 
for the assessment of writing skills as well as for the assessment of other indicators 
of preparation for higher education in the admissions process. Significantly, a 
sizable proportion of institutions assess writing skills at admission with timed 
writing exams which are developed by faculty or staff. In contrast, at institutions 
that use multiple choice writing exams in their admission process the 
preponderance of exams are prepared by a commercial agency. With regard to 
exemption, most (179, 70%) respondents indicated that their timed writing exams 
are developed by faculty or staff, although many (73, 28%) use commercially 
developed timed writing exams for exemption as well. It is likely that the scores 
from the latter are derived from the standardized tests that are traditionally used to 
make admissions decisions. Timed writing exams used for placement, entry, and 
graduation decisions are predominantly developed by faculty or staff at their own 
colleges or universities— by 273 (88 %) for placement, 228 (95 %) for proficiency, and 
101 (80 %) for graduation (see Table 11). 

Who Sets the Standards for the Evaluation of Timed Writing Samples? 

Data with regard to the individual who sets the standards at institutions 
using timed writing exams reflected the same pattern as did responses in the 
previous question, as well as for the parallel question regarding setting standards for 
multiple-choice exam.s. For the admissions process, for nearly half (43, 44%) the 
institutions, the decision comes from the "top down," or the administration, and 
slightly more than half (53, 54 %), from the faculty or staff. For the remaining 
purposes, standards are set predominantly by the faculty or staff — by 231 (88%) for 
exemption, 298 (93%) for placement, 237 (97%) for proficiency, and 102 (80%) for 
graduation (see Table 11). 

W ho Scores the Timed Writing Samples? 

The individu?ls who score the timed writing samples reflect the pattern of 
practices reported in the data regarding test development and standard setting. For 
admission, nearly half (42, 45 %) of the scoring is conducted by a commercial agency, 
as would be expected if they are standardized tests; however, many (37, 40%) are 
scored by English or composition faculty at their campuses (Table 10). Repeating the 
pattern observed in the related questions, scoring for exemption purposes is 
conducted by faculty at consid'Tably more institutions (193, 75%) than by a 
commercial agency (51, 20%). For the three other purposes, the faculty are 
predominantly involved in scoring the timed writing samples for placement 
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(264, 88%), proficiency (232, 97%), and graduation (93, 83%.). In addition, only a few 
respondents reported that they used a combination of two of the three approaches to 
scoring the timed writing samples — four, for admission; 10, for exemption; 19, for 
placement; 11, for proficiency; and 14, for graduation. 

Table 12 

Scoring Practices for Timed Writing Samples 



Who Scores Timed Writing Samples? 





Staff of 


a 


Eng. or 


comp. 








commercial 


faculty at 


Testing staff at 




agency 




own campus 


own 


campus 




# 


7r 


# 


7. 


# 




Admission 


42 


45 


37 


40 


14 


15 


Exemption 


51 


20 


193 


75 


15 


6 


Placement 


19 


6 


264 


88 


18 


6 


Proficiency 


2 


1 


232 


97 


6 


2 


Graduation 


lU 


9 


93 


83 


9 


8 



[Numbers and percentages reported are based on the number of institutions which 
responded to this particular item. Thus, the numbers reported for scoring writing 
samples may vary somewhat from the numbers reported for other items dealing 
with timed writing samples. Some respondents may have been unfamiliar with 
particular details of the assessment process.] 

Authority over the Assessment Process 

In addition to faculty participation in the assessment process, authority over 
the process is an issue of concern to educators. External testing programs may 
influcriLe what is taught and how it is taught (Haertel, 1988). Although college 
instructors may be less affected by pressure to shape their teaching and curriculum 
to fit external tests than their counterparts in K-12 schools, to some degree they too 
may sometimes face the choice of whether or not they should teach to the test. Like 
their K-12 counterparts, faculty in postsecondary institutions may face the 
diminution of their professional prerogatives, a loss of voice in determining what 
should be considered valuable enough to justify expending resources on assessing it. 
Moreover, externally mandated tests may lead to the inference that the instructor's 
judgment is not to be trusted (Haertel, 1988). Externally mandated tests are thus 
problematic because they can lead to loss of confidence in the faculty. 

To gather information relevant to these issues, the survey asked respondents 
to identify the primary governing authority over the assessment process for each 
purpose. In compliance with the survey directions, most respondents checked only 
one of the three possible responses. In order to reflect the pri nary practices, the 
responses of the few who checked more than one possibility were not counted. 



Table 13 

Governing Authority Over the Assessment Process 



By Whom Mandated? 

Dean College 





State/ 


Faculty 


or 




Trustees or 




Regie 


nal 


Member or 


Other 




University 




Authority 


Gn^up 




Administrator 


Regents 








# 




# 


7c 


# 7, 


Admisbion 


61 


23 


46 


17 


121 


49 


25 9 


Exemption 


16 


4 


261 


72 


6() 


17 


7 2 


Placement 


3M 




302 


71 


48 


11 


14 3 


Proficiency 


6 


2 


267 


88 


20 


T 
/ 


5 2 


Graduation 


25 


14 


m 


61 


23 


13 


18 10 



The results suggest that for most purposes, members of the faculty have authority 
over the assessment process. The exception appears to be the assessment of writing 
for making decisions about admission. Many institutions (121, 49%) report that an 
administrator mandates admission assessments, and in many cases, admission 
assessments are mandated by a state or regional authority (61, 23%). 



Consequences for Students 

In addition to the issues above, the survey also inquired about the 
consequences for students of decisions based on the assessment of writing skills, 
regardless of the particular method or combination of methods employed. In part, 
the interest of the Committee in this aspect of the assessment process stemmed from 
recent work in the development of assessment theory which anchors the concept of 
validity in the social context. Some measurement theorists (Messi<k, 1989) have 
begun to include consequences — the uses to which tests are put — as an integral 
feature of validity. In addition, the Committee included questions about 
consequences of assessment because of their concern that assessments should bene.i 
students, not penalize them unnecessarily. 

Table 14 
Consequences for Students 

For Placement 

Required at Optional at Not an Option at 





This 
# 


Campus 

7c 


This Campus 
# % 


This 
# 


Campus 

7r 


Take/pass Developmental English 
Course for Credit 


152 


54 


28 


10 


104 


37 


Take/pass Developmental 
English Course Without Credit 


221 


69 


37 


12 


63 


20 


Attend Tutoring 


^5 


32 


154 


52 


45 


15 


Retake Exam Until P«iss 


72 


28 


44 


17 


137 


54 



ERIC 



18 



I ) 



1 Percentages based on the number of institutions indicating they use writing assessment to make 
decisions about placement, N = 421 1 



For Proficiency for Entry into a Next Leve^ 

Required at Optional at Not an Option at 





Tliis Campus 


This Campus 


This 


Campus 




# 




# 




# 




Prepare a Portfolio and Appeal 




3 


58 


28 


144 




Take/Pass Designated English Course 














Without Credit 


218 


74 


34 


11 


44 


15 


Retake Exam Unlimited Times Until 














Pass 


51 


24 


27 


13 


137 


64 


Retake Exam Limited # of Times 


44 


22 


22 


11 


132 


67 



[Percentages based on number of institutions indicating they use writing assessment to make decision^:" 
about proficiency, N=3(H.l 



For Graduation 



Required at Optional at Not an Option at 

This Campus This Campus This Campus 





# 


7r 


# 


7 


# 


7 


Prepare a Portfolio and Appeal 


4 


3 


23 


19 


93 


78 


Take/Pass Designated English Course 














Without Credit 


70 


43 


37 


23 


54 


34 


Retake Exam Unlimited Times Until 














Pass 


46 


33 


21 


15 


73 


52 


Retake Exam Limited # of Times 


20 


18 


12 


11 


77 


71 



IPercentages based on the number of institutions indicating they use writing assessment to make 
decisions about graduation, N = 172.1 



Respondents answered five questions regarding the consequenc3s for 
students of decisions based on the assessment of writing skills, regardless of the 
particular method or combination of methods employed. That is, respondents 
answers referred to the consequences of any type of writing assessment, not to 
assessment conducted through a particular method. Questions in this section of the 
survey focused on placement, proficiency and graduation, partly, because it was felt 
that information relevant to these purposes would be of most use to faculty and 
others who are concerned with policy surrounding these issues within the college or 
university. In the interest of keeping the survey brief, questions about the 
consequences of writing assessment for decisions about admission and graduation 
were not included in the survey. 
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What are the Consequences of Decisions about Placement 



For campuses that assess writing skills for purposes of placement into 
freshman writing courses, students who fail to qualify for the regular freshman 
English course are required by most institutions (221, 69%) to take and pass a 
developmental English or writing course for which they receive no credit toward 
graduation (Table 14). On the other hand, many institutions (152, 53%) require 
students who failed the assessment to take a course for which they receive credit 
toward graduation. However, on many campuses students do not have either 
option — to take a course for credit (104, 37%) or a course for no credit (63, 20%). 
Many campuses offer optional opportunities to students who fail to qualify to attend 
tutoring (154, 52%) and on many campuses, tutoring is required (95, 32%). However, 
many do not provide students the option to retake the exam until their 
performance is satisfactory (137, 54%). 



What are the Consequences of Decisions about Proficiency ? 

Of campuses that assess for proficiency, most require students who fail the 
assessment to determine proficiency for entry into a next level to take a designated 
writing course (218, 74%). This approach appears to predominate, since many 
institutions do not even offer the other options. Many campuses do not allow 
students to prepare a portfolio (144, 69%), to retake the assessment an unlimited 
number of times until performance is satisfactory (137, 64%) or to retake the 
assessment a limited number of times (132, 67%) (See Table 14). 



What are the Consequences of Assessment for Graduation ? 

Of campuses that assess writing to make decisions abour graduation, most 
report that they do not provide students who fail the assessment the option of 
preparing a portfolio and appealing the decision, (93, 78%). Many of these campuses, 
however, require that students take and pass a designated writing course (70, 43%) or 
provide this as an option (37, 23%). In order to graduate, some campuses require (46, 
33%) or allow (21,15%) students to take the exam an unlimited number of times. 
Many campuses, however, do not allow students the options of retaking the exam 
an unlimited number of times (73, 52%) or a limited number of times (77, 71%). 



ERLC 



Respondent Opinions 

The three final questions on the survey elicited the opinions of respondents 
with respect to the assessment of writing skills at their institutions on a four-point 
scale ranging from "Agree Strongly," to "Disagree Strongly." Most respondents 
agreed (231, 51%) or agreed strongly (123, 23%) that the assessment of writing skills 
has had a big impact on writing instruction at their campuses. Further, most agreed 
(280, 52%>) or agreed strongly (125, 23%) that writing assessment has had a good 

or 



influence on writing instruction. Over half were satisfied with the writing 
assessment methods employed at their institutions; (59, 117o) agreed strongly, 
(226, 42%) agreed. However, a considerable number of respondents for their 
institutions either disagreed (159, 30%) or strongly disagreed (44, 8%) with the 
statement "I am satisfied with the methods used to assess writing at our campus." 
Thus, 38 percent of the respondents are unhappy with their present circumstances 
in the area of writing assessment (See Table 15). 



Table 15 

Respondent Opinions About the Assessment of Writing Skills 

(N=534) 



Agree 
Strongly 
# % 

The assessment of writing bkillb has 123 23 
had a big impact on writing 
iniitruction at our campus. 



Disagree No 

Ag ree Disa gree Strongly Response 

# 7r # ' % # 

231 43 94 18 27 5 59 11 



The assessment of writing skills has 23 280 52 58 11 16 3 55 10 

had a good mfluence on wnting 
instruction at our campus. 

I am satisfied with the methods ^26 42 159 30 44 8 46 9 

used to assess wnting at our campus. 



[Percentages based on total sample. N = 534] 



Summary for All Respondents 

The following discussion summarizes the predominant results of the survey 
in terms of the kinds of decisions made for specific purposes as reported by the total 
sample. Overall, 21 percent of the 534 responding institutions do not use writing 
assessment at all for the five different purposes surveyed — admission, exemption, 
plo-^ement, proficiency, and graduation. The major uses of writing assessment were 
for placement (79% of the sample using writing assessment) and exemptiori (71%) 
purposes. 

Admission 

°Of the sample of institutions using writing assessment, 43% used it for the 
admissions process. 

°Sixty-six percent use multiple choice assessments; 34 percent, timed writing 
samples. 

''Most writing assessments for this purpose are mandated by the dean or another 
administrator. 
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^Of the 66 percent using multiple choice, 

-the assessments are developed mainly by a commercial agency, 
-the standards are set by an administrator. 

'Of the 34 percent using timed writing samples, 

--topics are usually developed either by faculty or by a commercial agency, 
-standards are set mainly by faculty, but by an administrator for many of the 
assessments. 

-many are scored either by a commercial agency or by their own faculty, 
-many use either analytic or general impression scoring. 

-most do not give feedback to students, although many do so automatically or by 
request. 

^Most institutions do not encourage prewriting and /or revising in their writing 
assessments for purposes of admission. 

""Many institutions allow either 30 minutes or less or 31 to 60 minutes for students 
to complete the assessments. 

Exemption 

°Of the sample of institutions using writing assessment, 71 percent used it for the 
exemption process. 

°Forty-nine percent use multiple choice assessments; 51 percent timed writing 
samples. 

""Most writing assessments for this purpose are mandated by a faculty member or 
group. 

°Of the 49 percent using multiple choice, 

-the assessments are developed mainly by a commercial agency 
-standards set by their own faculty. 

""Of the 51 percent using timed writing samples, 

-topics are developed mainly by their faculty, many by a commercial agency, 
-standards are set mainly by their faculty. 

-they are scored mainly by their own faculty, but many are scored by a commercial 
agency. 

-many use either analytic or general impression scoring, 
-most give feedback to students, by request. 

""Most institutions do not encourage prewriting and/or revising in their writing 
assessments for decisions about exemption, although many do. 

^Many institutions allow 31 to 60 minutes for students to complete the assessments. 
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Placement 



^Of the sample of institutions using writing assessment, 79 percent used it for the 
placement process. 

^Forty-nine percent use multiple choice assessments; 51 percent, timed writing 
samples. 

''Most writing assessments for this purpose are mandated by a faculty member or 
group. 

"Of the 49 percent using multiple choice, 

-the assessments are developed mainly by a commercial agency 
--standards are set mainly by the faculty. 

°Of the 51 percent using timed writing samples, 
-topics are developed mainly by their faculty 
-standards are set mainly by their faculty 
-the writing samples are scored mainly by the faculty. 

—mainly, general impression scoring is used, although many use analytic scoring, 
-most institutions give feedback to students, by request, 
-many institutions encourage prewriting and/or revising in their writing 
assessments for decisions regarding placement, but many do not. 

''Many institutions allow 31 to 60 minutes for students to complete the assessments. 
Proficiency 

""Of the sample institutions using writing assessment, 57 percent used it to assess 
proficiency for entry into the next level of coursework. 

""Thirty-five percent use multiple choice assessments; 65 percent, timed writing 
samples. 

"^Most writing assessments for this purpose are mandated by a faculty member or 
group. 

°Of the 35 percent using multiple choice, 

-most of the assessments are developed by the faculty, although many are 

developed by a commercial agency, 
-standards are set mainly by the faculty. 

''Of the 65 percent using timed writing samples, 
-the topics are developed mainly by the faculty 
-standards are set mainly by the faculty, 
-the writing is scored mainly by the faculty. 
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--many institutions use either analytic or general impression scoring, 
-most give feedback to students automatically. 

^''Most institutions encourage prewriting and/or revising in their writing 
assessments for making decisions regarding proficiency. 

•'Most institutions allow 31 to 60 minutes for students to complete the writing; 
many allow 90 or more minutes, and some, 61-90 minutes. 



Graduation 



""Of the sample of institutions using writing assessment, 32 percent used it as a 
condition for graduation. 

^Thirty-eight percent use multiple choice assessments for this purpose; 62 percent, 
timed writing samples. 

°Most writing assessments for this purpose are mandated by a faculty member or 
group. 

°Of the 38 percent using multiple choice for this purpose, 

-most assessments are developed by the faculty, although many are developed by a 

commercial agency, 
-standards are set mainly by the faculty. 

°Of the 62 percent using timed writing samples for this purpose, 

-topics are developed mainly by the faculty 

-standards are set mainly by the faculty. 

-the writing samples are scored mainly by the faculty. 

-many use either analytic or general impression scoring. 

-most give feedback to students by request 

°Many institutions allow 31 to 60 minutes for students to complete the assessments; 
many allow 90 or more minutes, and some, 61-90 minutes. 
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SECTION IL Responses by Institutions of Different Sizes 



As noted in the introduction to this report, based on data suppUed by the United 
States Department of Education, the random sample drawn for the survey was stratified 
on the basis of size of the institution. To simplify reporting, the categories supplied by 
the department were collapsed into four categories: Extra Large (20,000+), Large (5,000- 
19,999), Medium (1,000 - 4,999), and Small (1-999). Of the institutions which responded, 
27 were extra large, 135 were large, 256 were medium and 116 were small. This section 
of the report provides results of responses to survey questions categorized by the size of 
the institution. 



Assessment Methods and Decisions Based on Assessment Results 

Which Decisions are Based on the Assessment of Writing Skills ? 

Of the 534 respondents, the numbers and percentages of those who indicated 
whether or not they based certain decisions, in whole or in part, on any type of 
assessment of writing skills were as follows (see Table 16): 

Table 16 

Decisions Based on Any Type of Assessment of Writing Skills by Size of Institution 

Extra Larg e Larg e Medium Small 

Yes No Yes No Yes No Yes No 





# 


7r 


# 


7r 


# 


7r 


# 


7r 


# 


7< 


# 


7r 


# 


7r 


# 


7 


Admission 


13 


48 


14 


52 


53 


41 


76 


59 


112 


46 


131 


53 


54 


53 


48 


47 


Exemption 


19 


73 


n 

/ 


27 


110 


84 


21 


16 


180 


74 


64 


26 


72 


75 


24 


25 


Placement 


21 


81 


5 


19 


116 


89 


15 


11 


199 


81 


46 


19 


H5 


84 


16 


16 


Proficiency 


13 


37 


10 


43 


86 


66 


45 


34 


148 


62 


91 


38 


57 


59 


40 


41 


Graduation 


8 


32 


17 


68 


40 


31 


88 


69 


83 


35 


156 


65 


41 


41 


59 


59 



Extra Large = 20,000+ 
Large = 5,000-19,999 
Medium = 1,IX)0- 4,999 
Small = 1-999 

{Percentages based on the # of institutions in each size category which responded to the question.] 



Following a similar pattern to that found for all respondents, for institutions 
which indicated that decisions on their campuses are based, in whole or in part, on any- 
type of assessment of writing skills, most use some form of writing assessment for 
placement and exemption decisions. Seventy-three percent of the extra large 
institutions, 84% of the large, 74% of the medium, and 75% of the small institutions 
report using some form of writing assessment for exemption purposes. Similarly, 81% 
of the extra large institutions, 89% of the large, 72% of the medium, and 84% of the 
small institutions report using some form of writing assessment for placement 
purposes. Relatively fewer institutions use some form of writing assessment for the 
other assessment purposes. A considerable number also use writing assessment to 
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make decisions regarding proficiency (57% of the extra large institutions, 66% of the 
large; 62% of the medium and 59% of the small). A fairly large number of institutions 
report using writing assessment as a condition of graduation. Thirty-two percent (8) of 
the extra large institutions, 31 % (40) of the large, 35% (83) of the medium and in 
proportion to institutions of other sizes, slightly more 41% (41) of the small institutions 
use some form of writing assessment as a condition of graduation. The results indicate 
that, overall, most of the decision-making based on writing assessment revolves around 
the freshman English course. 



What Methods are Used for the Different Assessment Purposes ? 

As noted in the section for all respondents, the second question focused on three 
particular assessment methods: multiple-choice exams, timed writing samples, and 
portfolios. Thus, several respondents who had indicated that they used writing 
assessment for certain purposes in question 1 did not respond to question 2. In 
addition, some respondents may have found this question complex and thus difficult to 
answer. An additional cross-check of respondents answers revealed that a few 
individuals who did not respond to question 2 later answered specific questions about a 
particular method. The results in this section, then, must be interpreted cautiously. 
Approximate numbers of institutions of different sizes using multiple choice exams, 
timed writing samples and/ or portfolios alone, or in combination with another method 
are reported in Table 17, The estimates for multiple choice and timed writing samples 
are based on the numbers of institutions who responded to specific questions regarding 
the development and implementation of those particular assessment methods. The 
estimates for portfolios are based upon the number of institutions reporting their use for 
each of the purposes in response to question 2. Because respondents at a number of the 
institutions which use writing assessment for particular decisions failed to respond to 
question 2, the numbers reported for portfolios may be an underestimate of their actual 
use. 
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Table 17 

Approximate Numbers of Institutions Using Multiple Choice Timed Writing Samples and/or 
Portfolios for Decision Making by iSize of Institution 



Admission 



Exemption 



Placement 



Proficiency 



Graduation 





Small 


Medium 


Large 


Extra Large 




# 




# 




# 7. 


# 7< 


Multiple Choice 


33 


61 


98 


88 


54 1(H) 


11 85 


Timed Writing 


24 


44 


43 


38 


30 57 


5 38 


Portfolio 


18 


33 


9 


8 


4 8 


1 8 




Small 




Medium 


Large 


Extra Large 




# 


7f 


# 


% 


# 9r 


# 7r 


Multiple Choice 


49 


68 


121 


67 


76 69 


13 68 


Timed Writing 


58 


81 


122 


68 


76 69 


15 79 


Portfolio 


7 


10 


23 


13 


12 11 


4 21 




Small 




Medium 


Large 


Extra Large 




# 




# 


Vo 


# 7c 


# 7r 


Multiple Choice 


64 


75 


154 


77 


85 73 


13 62 


Timed Writing 


71 


84 


153 


77 


82 71 


17 81 


Portfolio 


6 


7 


11 


6 


2 2 


3 14 




Small 




Medium 


Urge 


Extra Large 




# 


'7< 


# 


9f 


# 7r 


# 7f 


Multiple Choice 


26 


46 


70 


47 


30 35 


3 23 


Timed Writing 


44 


77 


118 


80 


70 81 


9 69 


Portfolio 


25 


44 


55 


37 


25 29 


5 38 




Small 


Medium 


Large 


Extra Large 




# 


% 


# 


% 


# 7c 


# 7c 


Multiple Choice 


20 


49 


37 


45 


18 45 


3 38 


Timed Writing 


30 


73 


56 


67 


32 80 


8 100 


Portfolio 


12 


29 


23 


28 


7 18 


3 38 



Responses indicate that of the institutions using multiple choice, timed writing 
samples and/ or portfolios to make decisions about admission, approximately 44% of 
the small institutions use multiple choice exams, compared to 65% of the medium, 61% 
of the large and 65% of the extra large institutions. Approximately 32% of the small 
institutions use timed writing samples, compared to 29% of the medium, 34% of the 
large and 29% of the extra large institutions. Approximately 24% of \he small 
institutions use portfolios, compared to only 6% of the medium, 5% of the large and 6% 
of the extra large institutions. 

Of the institutions using one of the three methods to make decisions about 
exemption, approximately 43% of the small institutions, 45% of the medium, 46% of the 
large and 387o of the extra large institutions use multiple choice exams. Approximately 
517o of the small institutions use timed writing samples, compared to 46% of the 



27 



medium, 46% of the large and 47% of the extra large institutions. Approximately 6% of 
the small institutions use portfolios, compared to 9% of the medium, 7% of the large and 
13% of the extra large institutions. 

Of the institutions using one or more of the three methods to make decisions 
about placement, approximately 45% of the small institutions use multiple choice 
exams, compared to 48% of the medium, 50% of the large and 39% of the extra large 
institutions. Approximately 50% of the small ^^<^titutions use timed writing samples, 
compared to 48% of the medium, 49% of the large and 52% of the extra large 
institutions. Approximately 4% of the small institutions use portfolios, compared to 37o 
of the medium, 1% of the large and 9% of the extra large institutions. 

Of the institutions using one or more of the three methods to make decisions 
about proficiency, approximately 27% of the small institutions use multiple choice 
exams, compared to 29% of the medium, 24% of the large and 18% of the extra large 
institutions. Approximately 46% of the small institutions use timed writing samples, 
compared to 49% of the medium, 56% of the large and 53% of the extra large 
institutions. Approximately 26% of the small institutions use portfolios, compared to 
23%) of the medium, 20% of the large and 29% of the extra large institutions. 

Of the institutions using one or more of the three methods to make decisions 
about graduation, approximately 32% of the small institutions use multiple choice 
exams, compared to 32% of the medium, 32% of the large and 21% of the extra large 
institutions. Approximately 48% of the small institutions use timed writing samples, 
compared to 48% of the medium, 56% of the large and 57% of the extra large 
institutions. Approximately 19% of the small institutions use portfolios, compared to 
20%) of the medium, 12% of the large and 21% of the extra large institutions. 

Practices Associated with Timed Writing Samples 

As noted in the section for all respondents, several of the survey questions 
focused on the assessment process, asking who develops, scores, and/or makes 
decisions about the results of particular exams, how particular exams are implemented 
and scored, and what kinds of feedback students receive. 

How Many Samples are Collected in Timed Writing Assessments ? 

Table 18 

Numbers of Writing Samples Collected by Size of Institution 

Admission 





Small 




Medium 


Uirge 




Extra Large 




# 


% 


# 


% 


# 


% 


# % 


One 


14 


82 


19 


76 


10 


91 


2 67 


Two 


2 


12 


1 


4 


1 


9 


1 33 


Three 


0 


0 


1 


4 


0 


0 


0 0 


Four or More 


1 


6 


4 


16 


0 


0 


0 I) 



3 7 
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Exemption 



One 
Two 
Three 
Four or More 

PK^ cement 



One 
Two 
Three 
Four or More 

Proficiency 



One 
Two 
Three 
Four or More 

Graduation 



One 
Two 
Three 
Four or More 



Small Medium 



# 7r 


# 7- 


36 84 


71 78 


4 9 


9 10 


1 2 


5 5 


2 5 


6 7 


Small 


Medium 


# 7c 


# 7, 


44 75 


103 82 


h U) 


8 6 


2 3 


0 0 


7 12 


14 11 


Small 


Medium 


# 7r 


# 7r 


17 50 


51 56 


4 12 


10 11 


1 3 


7 8 


12 35 


23 25 


Small 


Medium 


# 7, 


# 7r 


16 84 


26 72 


1 5 


4 11 


1 5 


2 6 


1 5 


4 11 



Large 


Extra Large 


# 7r 


# 7 


79 


9 75 


5 11 


3 25 


3 6 


.0 0 


2 4 


0 0 


Large 


Extra Lirge 


# 7r 


# 7r 


45 69 


13 93 


7 11 


0 0 


3 5 


1 7 


10 15 


0 0 


Large 


Extra Large 


# 7r 


# 7 


29 63 


6 100 


6 13 


0 0 


1 2 


0 0 


10 22 


0 0 


Large 


Extra Large 


# 7 


# 


19 95 


5 1(K) 


1 5 


0 0 


0 0 


0 0 


0 0 


0 0 



Of the institutions indicating that timed writing samples are collected to make 
decisions about admission, approximately 82% (14) of the small institutions rely on a 
single sample, compared to 76% (19) of the medium, 91% (10) of the large, and 67% (2) 
of the extra large. For decisions about exemption, approximately 84% (36) of the small 
institutions rely on a single sample, compared to 78% (71) of the medium, 79% (34) of 
the large, and 75% (9) of the extra large. For decisions about placement, approximately 
75% (44) of the small institutions rely on a single sample, compared to 83% (103) of the 
medium, 69% (45) of the large, and 93% (13) of the extra large. For decisions about 
proficiency, approximately 50% (17) of the small institutions rely on a single sample, 
compared to 56% (51) of the medium, 63% (29) of the large, and 100% (6) of the extra 
large. For decisions about graduation, approximately 84% 16) of the small institutions 
rely on a single sample, compared to 72% (26) of the medium, 95% (19) of the large, and 
100% (5) of the extra large. For most assessment purposes, the patterns of responses 
appear to be similar, regardless of the size of the institution. In the case of assessment 
for proficiency, proportionately more of the extra large institutions appear to rely on a 
single sample. The reader should note, however, that the percentages for extra large 
institutions are based on a small number of responses since there were few institutions 
in this size category. 



How Much Time Is Allowed for Writing in Timed Writing Assessments ? 



As noted in the section for all respondents, the amount of time students are 
allowed to complete writing samples is an issue of concern to professionals who want to 
see a closer match between assessment procedures and teachers' views of effective 
pedagog}^ in which students are encouraged to plan and revise their writing. 
Assessments which allow brief amounts of time tap only first draft writing. To gather 
information about this issue, the survey asked respondents who used timed writing 
samples to make assessment decisions to indicate the amount of time allo\A;ed per 
sample. The results for institutions of different sizes are presented in Table 19. 

Table 19 

Average Times Allowed to Complete a Writing Sample by Size of Institution 

Admission 







Sinall 


Medium 




Large 


Extra Large 




# 




# 




# 


% 


# 7c 


<3l) 


5 


31 


13 


43 


3 


23 


1 25 


31-60 


6 


38 


12 


40 


7 


54 


2 50 


61-90 


3 


19 


3 


10 


1 


8 


1 25 


90+ 


2 


13 


2 


7 


2 


15 


0 0 



Exemption 





Small 




Medium 


Large 


Extra Large 




# 




# 




# 


7c 


# % 


<30 


7 


16 


16 


15 


7 


12 


1 7 


31-60 


25 


56 


45 


42 


23 


39 


8 57 


61-90 


7 


16 


26 


24 


12 


20 


0 0 


90+ 


6 


13 


20 


19 


17 


29 


5 36 



Placement 

Small Medium Large Extra Large 





# 


% 


# 


% 


# 


7c 


# 


7c 


<30 


16 


25 


30 


20 


13 


16 


2 


11 


31-60 


36 


55 


79 


54 


49 


60 


11 


58 


61-90 


6 


9 


25 


17 


11 


13 


2 


11 


90+ 


7 


11 


13 


9 


9 


11 


4 


21 



Proficien cy 





Small 




Medium 


Ur 


ge 


Extra Large 




# 


% 


# 


% 


# 


% 


# % 


<30 


6 


15 


10 


9 


2 


3 


0 0 


31-60 


18 


44 


50 


45 


20 


34 


5 56 


61-90 


7 


17 


22 


20 


16 


28 


0 0 


90+ 


10 


24 


30 


27 


22 


38 


4 44 



Graduation 





Small 




Medium 




Large 


Extra Large 




# 




# 


% 


# 


% 


# 7c 


<30 


3 


12 


3 


6 


0 


0 


0 0 


31-60 


10 


38 


20 


41 


12 


46 


1 17 


61-90 


3 


12 


8 


16 


(^ 


23 


1 17 


90+ 


10 


38 


18 


37 


8 


31 


4 67 



30 



3^ 



Of the institutions indicating they use timed writing samples in assessment to 
make decisions about admission, approximately 31% (5) of the small institutions allow 30 
minutes or less, compared to 43% (13) of the medium, 23% (3) of the large, and 259b (1) 
of the extra large institutions. Approximately 38% (6) of the small institutions allow 31 
to 60 minutes, compared to 407o (12) of the medium, 54% (7) of the large, and 50% (2) of 
the extra large institutions. Approximately 197o (3) of the small institution allow 61 to 
90 minutes per sample, compared to 10% (3) of the medium, 8% (1) of the large, and 
23% (1) of the extra large institutions. Approximately 13% (2) of the small institutions 
allow more than 90 minutes, compared to 7% (2) of the medium, 15% (2) of the large, 
and none of the extra large institutions. 

Of those using timed writing samples to make decisions about exemption, 
approximately 167o (7) of the small institutions allow 30 minutes or less, compared to 
15% (16) of the medium, 127o (7) of the large, and 7% (1) of the extra large institutions. 
Approximately 56% (25) of the small institutions allow 31 to 60 minutes, compared to 
427o (45) of the medium, 39%> (23) of the large, and 57% (8) cf the extra large institutions. 
Approximately 16% (7) of the small institutions allow 61 to 90 minutes per sample, 
compared to 24% (26) of the medium, 20% (12) of the large, and none of the extra large 
institutions. Approximately 13% (6) of the small institutions allow more than 90 
minutes, compared to 19% (20) of the medium, 29% (17) of the large, and 36% (5) of the 
extra large institutions. 

Of those using timed writing samples to make decisions about placement, 
approximately 25% (16) of the small institutions allow 30 minutes or less, compared to 
20% (30) of the medium, 16% (13) of the large, and 11% (2) of the extra large institutions. 
Approximately 55% (36) of the small institutions allow 31 to 60 minutes, compared to 
54% (79) of the medium, 60% (49) of the large, and 58% (11) of the extra large 
institutions. Approximately 9% (6) of the small institutions allow 61 to 90 minutes per 
sample, compared to 17% (25) of the medium, 13% (11) of the large, and 11% (2) of the 
extra large institutions. Approximately 11% (7) of the small institutions allow more 
than 90 minutes, compared to 9% (13) of the mediuirt, 11% (9) of the large, and 21% (4) 
of the extra large institutions. 

Of those using timed writing samples to make decisions about proficiency, 
approximately 15% (6) of the small institutions allow 30 minutes or less, compared to 
97o (10) of the medium, 3% (2) of the large, and none of the extra large institutions. 
Approximately 44% (18) of the small institutions allow 31 to 60 minutes, compared to 
457o (50) of the medium, 34% (20) of the large, and 567o (5) of the extra large institutions. 
Approximately 17% (7) of the small institutions allow 61 to 90 minutes per sample, 
compared to 20% (22) of the medium, 28% (16) cr the large, and none of the extra large 
institutions. Approximately 24% (10) of the small institutions allow more than 90 
minutes, compared to 27% (30) of the medium, 38% (22) of the large, and 44% (4) of the 
extra large institutions. 

Of those using timed writing samples to make decisions about graduation, 
approximately 12% (3) of the small institutions allow 30 minutes or less, compared to 
67o (3) of the medium, none of the large, and none of the extra large institutions. 
Approximately 38% (10) of the small institutions allow 31 to 60 minutes, compared to 
41% (20) of the medium, 46% (12) of the large, and 17% (1) of the extra large institutions. 
Approximately 12% (3) of the small institutions allow 61 to 90 minutes per sample, 
compared to 167o (8) of the medium, 23% (6) of the large, and 17% (1) of the extra large 




institutions. Approximately 38% (10) of the small institutions allow mc r j than 90 
minutes, compared to 37% (18) of the medium, 31% (8) of the large, and 67% (4) of the 
extra large institutions. 

Are Prewriting and/or Revising Activities Encouraged When Timed Writing Samples 
Are Collected? 



In order to gather information about the degree to which assessment procedures 
mirrored typical patterns in writing instruction, respondents were also asked to indicate 
whether or not they encourage prewriting and/ or revising activities during timed 
writing tasks. 

Table 20 

Prewriting and/or Revising Activities in the Collection of 
Timed Writing Samples by Size of Institution 



Admission 



Yes 
No 



Small 
# % 
6 26 
17 74 



Medium 
# % 
6 21 
28 79 



Large 
# 7c 
4 17 
19 83 



Extra Large 

# 7r 

3 75 
1 25 



Exemption 

Small Medium Ltirge Extra Large 





# 


% 


# 


% 


# 


% 


# 


7f 


Yes 


25 


45 


47 


45 


26 


40 


9 


64 


No 


31 


55 


58 


55 


39 


60 


5 


36 



Placement 

Small Medium Lirge Extra Large 





# 


'7r 


# 


7o 


# 


'7c 


# 


% 


Yes 


37 


51 


79 


51 


41 


52 


12 


71 


No 


35 


49 


76 


49 


38 


48 


5 


29 



Proficiency 

Small Medium Large Extra Large 





# 


% 


# 


% 


# 


7o 


# 


% 


Yes 


30 


67 


86 


69 


48 


76 


9 


82 


No 


15 


33 


38 


31 


15 


24 


2 


18 



Graduatio n 

Small Medium Large Extra Large 





# 


% 


# 


% 


# 


% 


# 


% 


Yes 


19 


66 


38 


66 


17 


57 


5 


71 


No 


10 


34 


20 


34 


13 


43 


2 


29 



Overall, the patterns of responses categorized by size of institutions mirror the 
patterns found for all respondents together. That is, there did not seem to be any 
important variation from the pattern according to the size of the institution. Of the 
institutions responding to this question, 26% (6) of the small institutions, 21% (6) of the 
medium, 17% (4) of the large, and 75% (3) of the extra large institutions indicated that 
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they did encourage prewriting and/or revising activities when timed writing samples 
were collected for admissions purposes. When timed samples were collected for 
exemption purposes, 45% (25) of the small institutions, 45% (47) of the medium, 40% 
(26) of the large, and 64% (9) of the extra large institutions indicated they encouraged 
these kinds of process activities. When samples were collected for purposes of 
placement, 51% (37) of the small, 51% (79) of the medium, 52% (41) of the large, and 64 7o 
(9) of the extra large institutions indicated ihey encouraged these kinds of process 
activities. When samples were collected as a condition of graduation, 66% (19) of the 
small, 66% (38) of the medium, 57% (17) of the large, and 71% (5) of the extra large 
institutions indicated they encouraged such activities. 

Is Feedback Provided to Students about the Results of Timed Writing Assessments? 

As noted above, an issue in assessment procedures is the question of whether 
students should receive feedback about their performance. For pragmatic reasons, 
(expense, time) some institutions may not provide feedback. However, some educators 
argue that assessment should contribute to learning and that feedback is thus a 
necessary part of the assessment process. To gather information relevant to this issue, 
one question on the survey asked whether students had to receive feedback about their 
performance when timed writing samples were collected. 

Table 21 

Feedback to Students About Perfonnance on Timed Writing Sample(s) by Size of Institution 

Admission 





Small 


Medium 


Large 


Extra Large 




# % 


# % 


# 7r 


# % 


Yes, automatic 


7 28 


15 39 


5 18 


4 67 


Yes, if request 


9 36 


10 26 


5 18 


0 0 


No, no feedback 


9 36 


13 34 


18 64 


2 33 



Exemption 





Small 




Medium 


Large 


Extra Large 




# 


% 


# 


% 


# % 


# % 


Yes, automatic 


15 


27 


35 


29 


23 33 


5 33 


Yes, if request 


31 


56 


63 


53 


26 37 


6 40 


No, no feedback 


9 


16 


22 


18 


21 30 


4 27 


Placement 
















Small 


Medium 


Large 


Extra Large 




# 


% 


# 


% 


# % 


# % 


Yes, automatic 


21 


30 


57 


35 


27 31 


8 42 


Yes, if request 


36 


51 


79 


49 


40 47 


8 42 


No, no feedback 


13 


19 


25 


16 


19 22 


3 16 



'I 3^ 



Proficiency 



Yes, automatic 
Yes, if request 
No, no feedback 



Sniall 


Medium 


Large 


Extra Large 


# 7f 


# 




# '7i 


# % 


19 44 


64 


53 


33 49 


4 33 


19 44 


50 


42 


31 46 


6 50 


5 12 


6 


5 


4 6 


2 17 



Graduation 





Small 




Medium 


Large 


Extra 


Large 




# 


7r 


# 


7r 


# 7r 


# 


7^ 


Yes, automatic 


8 


30 


26 


46 


8 27 


2 


25 


Yes, if request 


19 


70 


28 


49 


21 70 


4 


50 


No, no feedback 


0 


0 


3 


5 


1 3 


2 


25 



Of the institutions responding to this question, 36% (9) of the small, 34% (13) of 
the medium, and 33% (2) of the extra large institutions report that they do not provide 
feedback to students about their performance on timed writing samples during the 
admissions process. In contrast, 64% of the large institutions indicate that they do not 
provide such feedback. 

In the case of assessments conducted for exemption purposes, 16% (9) of the 
small institutions, 18% (22) of the medium, 30% (21) of the large and 27% (4) of the extra 
large institutions report that they do not provide feedback. In the case of assessments 
conducted for placement purposes, 19% (13) of the small institutions, 16% (25) of the 
medium, 22% (19) of the large, and 16% (3) of the extra large institutions report that 
they do not provide feedback. 



Participants in the Assessment Process 

As noted in the introduction, some professionals argue for the value of direct 
involvement of faculty in the assessment process. They point out that when faculty are 
involved in development, assessments are more likely to be grounded in the faculty's 
"theories, curricula, and classroom practices" (Greenberg, 1992, 15). It has also been 
argued that participation in the assessment process can serve as a vehicle for effective 
staff development, with the caution that "a good test can have as positive an effect on 
mediocre teaching as a bad test has negative effects on teaching" (Lucas, 1988, 2). To 
gather information about the extent of faculty involvement in the writing assessment 
process in postsecondary institutions, specific questions about faculty involvement were 
included in the survey. 

Who Develops Multiple Choice Exams ? 

Following the pattern for all respondents, among the institutions who use 
multiple choice exams, most exams are developed by the staff of a commercial agency 
(such as the ACT, SAT, or CLEP) for the assessment of writing at the pre-freshman and 
freshman levels for admission, regardless of the size of the institution. For admission, 
88% of the small institutions, 96% of the medium, 90% of the large, and 100% of the 
extra large institutions which responded to this question employ a commercial agency's 
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exams. As noted previously, this finding is not surprising, since standardized tests are 
comnnonly used, particularly for admission and exemption (such as the AP English 
examination). Evidently many institutions also extend the application of these scores to 
placement in freshman English or writing courses. Most institutions, regardless of size, 
report using a commercial exam for these purposes (see table 21). However, following 
the pattern for all respondents, of those using multiple choice exams to assess writing 
proficiency for entry to the next level of coursework, the multiple choice exams are 
developed at most institutions by their own faculty or staff, at small institutions (15, 
60%), at medium (35, 53%), at large institutions (17, 63%) and at the extra large 
institutions, (2, 67%). 

Table 22 

Participants in the Development of Multiple Choice Exams By Size of Institution 
Commercial Agency Staff 





Small 




Medium 


La 


rge 


Extra Large 




# 


'7c 


# 


7c 


# 


% 


# 7r 


Admission 


29 


88 


90 


95 


46 


90 


11 100 


Exemption 


36 


77 


95 


81 


57 


78 


10 77 


Placement 


44 


72 


117 


82 


53 


69 


9 69 


Proficiency 


9 


36 


28 


42 


5 


19 


1 33 


Graduation 


7 


35 


11 


31 


5 


28 


1 33 



Faculty /Staff at Own College or University 





Small 




Medium 


Large 




Extra Large 




# 


% 


# 


% 


# 


% 


# 7r 


Admission 


1 


3 


4 


4 


2 


4 


0 0 


Exemption 


10 


21 


19 


16 


9 


12 


1 8 


Placement 


12 


20 


19 


13 


12 


16 


2 15 


Proficiency 


15 


60 


35 


53 


17 


63 


2 67 


Graduation 


10 


50 


20 


57 


5 


28 


1 33 



State Agency/Dept. Staff 

Small Medium Large Extra Large 





# 


7c 


# 


% 


# 


% 


# 


7c 


Admission 


3 


9 


0 


0 


3 


6 


0 


0 


Exemption 


1 


2 


3 


3 


7 


10 


0 


0 


Placement 


5 


8 


6 


4 


12 


16 


0 


0 


Proficiency 


1 


4 


3 


5 


5 


19 


0 


0 


Graduation 


3 


15 


4 


11 


8 


44 


1 


33 



Who Sets the Standards for Multiple Choice Exams? 

The individual who sets the standards (e.g., who decides which scores are 
acceptable) at institutions using multiple choice writing assessments for the purpose of 
admission is predominantly the administrator at the college or university, at small 
institutions (15, 52%), at medium, (58, 64%), at large (30, 61%) at extra large (5, 63%). 
Following the pattern for all respondents, a relatively large number of institutions 
report involving their faculty, (13, 45%) at small institutions but relatively fewer at 
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medium (27, 307o), large (13, 27%) and extra large institutions (3,38%). As would be 
expected, faculty are typically not very active in the admissions process, except perhaps, 
at small colleges. For all other purposes, however, the faculty at most institutions set 
the standards, regardless of the size of the institution (see Table 23). 

Table 23 

Participants in Setting the Standards for Multiple Choice Exams by Size of Institution 



Cc^mmercial Aeency Staff 





Small 




Medium 


Large 


Extra Large 




# 




# 


% 


# 




# % 


Admission 


15 


52 


58 


64 


30 


61 


5 62 


Exemption 


8 


19 


22 


19 


13 


19 


3 30 


Placement 


7 


11 


18 


13 


10 


13 


0 0 


Proficiency 


2 


6 


5 


7 


2 


6 


0 0 


Graduation 


3 


13 


5 


12 


3 


17 


0 0 



Faculty/Staff at Own College or University 





Small 




Medium 


Large 


Extra Large 




# 




# 


% 


# 


% 


# % 


Admission 


13 


45 


27 


30 


13 


27 


3 38 


Exemption 


34 


79 


86 


75 


48 


71 


7 70 


Placement 


51 


84 


114 


82 


59 


78 


10 100 


Proficiency 


29 


94 


68 


91 


27 


87 


4 100 


Graduation 


17 


74 


33 


79 


11 


61 


1 50 



State Agency /Dept. Staff 

Small 





# 


% 


Admission 


1 


3 


Exemption 


1 


2 


Placement 


3 


5 


Proficiency 


0 


0 


Graduation 


3 


13 



Medium Large Extra Large 



# 


% 


# 


% 


# 


7r 


6 


7 


6 


12 


0 


0 


7 


6 


7 


10 


0 


0 


7 


5 


7 


9 


0 


0 


2 


3 


2 


6 


0 


0 


4 


10 


4 


22 


1 


50 



Who Develops the Topics for Timed Writing Samples? 

- In contrast to the results for multiple choice exams, among the institutions that 
use timed writing samples for their assessment purposes, most of the assessments were 
developed by faculty or staff at their own colleges or universities. The one exception is 
for admission-nearly half the respondents at medium sized institutions (21, 50%) and 
large institutions (14, 50%) used topics for timed writing samples developed by a 
commercial agency (3, 75% at the extra large institutions. At small institutions, in 
contrast, most (17, 71%) were developed by faculty or staff at their own institution. 
These data suggest, as previously mentioned, that the commonly used standardized 
tests which include a writing sample provide scores for the assessment of writing skills 
as well as for the assessment of other indicators of preparation for higher education in 
the admissions process at most institutions, but also, that relatively more of the small 
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institutions may include exams developed on their own campuses in the admissions 
process. 

With regard to exemption, most respondents indicated that their timed writing 
exams are developed by faculty or staff, regardless of the size of the institutions. Timed 
writing exams used for placement, entry, and graduation decisions are also 
predominantly developed by faculty or staff at their own colleges or universities, 
regardless of the institutions size (see Table 23). 



Table 24 

Participants in the Development of Timed Writing Exams by Size of Institutions 

C(TOmercial Agency Staff 





Small 




Medium 


Large 


Extra Large 




# 


ST 


# 


'I 


# 




# 7r 


Admission 


4 


16 


21 


50 


14 


50 


3 75 


Exemption 


16 


27 


29 


24 


24 


35 


4 31 


Placement 


7 


10 


14 


9 


5 


6 


0 0 


Proficiency 


1 


2 


7 


6 


0 


0 


0 0 


Graduation 


3 


10 


4 


7 


2 


6 


0 0 



Faculty/Staff at Own University 





Small 




Med 


ium 


Large 


Extra Large 




# 




# 


7r 


# 


7r 


# 




Admission 


17 


71 


20 


48 


13 


46 


1 


25 


Exemption 


41 


71 


86 


72 


43 


63 


9 


69 


Placement 


60 


86 


131 


89 


67 


87 


15 


94 


Proficiency 


42 


95 


110 


94 


67 


97 


9 


UK) 


Graduation 


25 


83 


46 


82 


23 


74 


7 


88 



State Agency /Dept. Staff 

Small 





# 


% 


Admission 


3 


13 


Exemption 


1 


2 


Placement 


3 


4 


Proficiency 


1 


2 


Graduation 


2 


7 



Medium 




Urge 


Extra Large 


# % 


# 


% 


# % 


1 2 


1 


4 


0 0 


4 3 


1 


1 


0 0 


3 2 


5 


6 


1 6 


0 0 


2 


3 


0 0 


6 11 


6 


19 


1 12 



Who Sets the Standards for the Evaluation of Timed Writing Samples? 

Data with regard to the individual who sets the standards at institutions using 
timed writing exams reflected the same pattern as did responses in the previous 
question, as well as for the parallel question regarding setting standards for multiple- 
choice exams. For the admissions process, at rnany of the institutions, the decision 
comes from the "top down," or the administration, at small institutions (7, 32%), at 
medium (19, W/o) at large (14, 50%) and at extra large (3, 60%). Many of the standards, 
however, are set by faculty or staff, at small institutions, (15,68%) at medium (19,44%) at 
large (14, 50%) and at extra large (3, 60%). For the remaining purposes, standards are set 
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predominantly by the faculty or staff, regardless of the size of the institution (see Table 
24). 



Table 25 

Participants in Setting the Standards for Timed Writing Exams 



Commercial Agency Staff 





Small 




Medium 


Large 


Extra Large 




# 


'I 


# 


7r 


# 




# 7r 


Admission 


/ 


32 


19 


44 


14 


50 


3 6(1 


Exemption 


6 


11 


8 


7 


5 


7 


1 6 


Placement 


5 


7 


4 


3 


1 


1 


0 0 


Proficiency 


2 


4 


2 


2 


2 


3 


(i 0 


Graduation 


2 


7 


3 


5 


3 


10 


1 11 



Faculty /Staff at Own University 





Small 




Med 


ium 


Large 


Extra 


Large 




# 




# 


% 


# 


'7c 


# 


7r 


Admission 


15 


68 


24 


56 


12 


43 


2 


40 


Exemption 


49 


88 


108 


89 


59 


82 


15 


94 


Placement 


64 


89 


147 


94 


70 


93 


17 


100 


Proficiency 


43 


96 


119 


97 


64 


96 


11 


100 


Graduation 


24 


83 


49 


83 


22 


73 


7 


78 



State Agency/Dept. Staff 





Small 




Medium 


Large 




Extra Large 




# 


'I 


# 


% 


# 




# 7r 


Admission 


I) 


t) 


0 


0 


2 


7 


0 0 


Exemption 


1 


1 


5 


4 


4 


6 


0 0 


Placement 


3 


4 


5 


3 


4 


6 


0 0 


Proficiency 


0 


0 


1 


1 


1 


1 


0 0 


Graduation 


3 


10 


7 


12 


5 


17 


1 11 



Who Scores the Timed Writing Samples? 

The individuals who score the timed writing samples reflect the pattern of 
practices reported in the data regarding test development and standard setting. For 
admission, nearly half (42, 45 %) of the scoring is conducted by a commercial agency at 
most institutions, as would be expected if they are standardized tests, (21, 54%) at 
medium sized institutions, (15, 56% )at large, and (3, 60%) at extra large. The exception 
is the small colleges, where only (3, 14%) of the writing assessments are reported to be 
scored by a commercial agency, (13, 59%) by faculty and staff at their own university, 
and (6, 27%) by staff of a state agency or department. For the other purposes, most 
limed writing samples are scored by faculty or staff at their own institution, regardless 
of institution size. 
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Table 26 



Scoring Practices for Timed Writing Samples by Size of Institution 
Commercial Agency Staff 





Small 




Medium 


Large 


Extra Large 




# 




# 




# 




# 7r 


Admission 


3 


14 


21 


54 


15 


56 


3 60 


Exemption 


11 


20 


22 


18 


16 


23 


2 13 


Placement 


6 


9 


8 


5 


5 


7 


0 0 


Proficiency 


1 


2 


1 


1 


0 


0 


0 0 


Graduation 


3 


10 


4 


9 


3 


11 


0 0 



Faculty /Staff at Own University 





Small 




Med 


ium 


Urge 




Extra 


Urge 




# 




# 


% 


# 


% 


# 


% 


Admission 


13 


59 


14 


36 


8 


30 


2 


40 


Exemption 


41 


73 


92 


77 


48 


70 


12 


80 


Placement 


56 


86 


129 


88 


63 


86 


16 


94 


Proficiency 


42 


93 


114 


97 


66 


97 


10 


100 


Graduation 


23 


77 


40 


85 


24 


86 


6 


86 



State Agency/Dept. Staff 





Small 




Medium 




Large 


Extra Large 




# 




# 


% 


# 


% 


# % 


Admission 


6 


27 


4 


10 


4 


15 


0 0 


Exemption 


4 


7 


5 


4 


5 


7 


1 7 


Placement 


3 


5 . 


9 


6 


5 


"7 


1 6 


Proficiency 


2 


4 


2 


2 


2 


3 


0 0 


Graduation 


4 


13 


3 


6 


1 


4 


1 14 



Authority over the Assessment Process 

As noted in the introduction to this report, authority over the assessment process 
is an issue of concern to educators because externally mandated testing programs may 
usurp faculty authority and diminish their professional prerogatives. To gather 
information relevant to this issue, the survey asked respondents to identify the primary 
governing authority over the assessment process for each assessment purpose. Table 27 
presents the results categorized by size of institution. 
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Table 27 

Governing Authority Over the Assessment Process by Size of Institution 



Admission 



State/Regional Authority 
Faculty 

Dean or other administrator 
Trustees or Regents 



Small Medium 

5 11) 31 26 

13 27 21 17 

27 55 61 50 

4 8 8 7 



Large Extra Large 



# 


7r 


# '7, 


23 


32 


2 18 


8 


11 


4 36 


29 


41) 


4 36 


12 


17 


1 9 



Exemption 

State /Regional Authority 
Faculty 

Dean or other administrator 
Trustees or Regents 



Small Medium 



# 


7r 


# 


7c 


0 


0 


9 


5 


50 


75 


129 


78 


17 


25 


25 


15 


0 


0 


2 


1 



Large Extra Large 



# 


7f 


# 


7r 


6 


6 


1 


6 


68 


72 


14 


82 


18 


19 


0 


0 


3 


3 


2 


12 



Placement 



State /Regional Authority 
Faculty 

Dean or other administrator 
Trustees or Regents 



Small 


Medium 


Large 




Extra Large 


# 7c 


# 


7< 


# 


7 


# 7r 


4 5 


18 


9 


12 


11 


4 19 


58 75 


148 


77 


82 


74 


14 67 


13 17 


24 


12 


9 


8 


2 10 


2 3 


3 


2 


8 


7 


1 1 



Proficiency 



State /Regional Authority 
Faculty 

Dean or other administrator 
Trustees or Regents 



Graduation 



State /Regional Authority 
Faculty 

Dean or other adnninistrator 
Trustees or Regents 



Small Medium 



# '7f 


# <7r 


1 2 


3 2 


46 84 


142 92 


6 n 


8 5 


2 4 


1 1 


Small 


Medium 


# '7c 


# % 


5 13 


13 16 


24 62 


53 65 


7 18 


10 12 


3 8 


6 7 



Large 


Extra Large 


# 7f 


# 7, 


2 3 


0 0 


68 87 


11 1(K) 


6 8 


0 0 


2 3 


0 0 


Large 


Extra Large 


# 7c 


# 7, 


6 13 


1 13 


27 59 


5 63 


6 13 


0 0 


7 15 


2 25 



Respondents' reports indicate that 10% (5) of writing assessments at small 
institutions for admission purposes are mandated by external authorities, 26% (31) at 
medium institutions, 32% (23) at large, and 18% (2) at extra large. Following the pattern 
for all respondents, faculty appear to have more authority in assessments conducted for 
other purposes. Most assessments for exemption are mandated by a faculty member or 
group, 75% (50) at small institutions, 78% (129) at medium, 72% (68) at large, and 827o 
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(14) at extra large institutions. Most assessments for placement are mandated by a 
faculty member or group, 75% (58) at small institutions, 77% (48) at medium, 74% (82) at 
large, and 67% (14) at extra large institutions. Most assessments for proficiency are 
mandated by a faculty member or group, 84% (46) at small institutions, 927o (142) at 
medium, 87% (68) at large, and 100% (11) at extra large institutions. Several 
assessments for graduation, however, are mandated by a state or regional authority, 
13% (5) at small institutions, 167o (13) at medium, 13% (6) at large, and 13% (1) at extra 
large institutions. In addition, several assessments for placement are mandated by a 
state or regional authority, 57o (4) at small institutions, 9% (18) at medium, U7o (12) at 
large, and 19% (4) at extra large institutions. 



Respondent Opinions 

As noted in the introduction, the three final questions on the survey elicited the 
opinions of respondents with respect to the assessment of writing skills at their 
institutions on a four-point scale ranging from "Agree Strongly," to "Disagree Strongly." 
Most respondents agreed or agreed strongly that the assessment of writing skills has 
had a big impact on writing instruction at their campuses. Further, most agreed or 
agreed strongly that writing assessment has had a good influence on writing 
instruction. Over half were satisfied with the writing assessment methods employed at 
their institutions. However, a considerable number of respondents for their institutions 
either disagreed or strongly disagreed with the statement "I am satisfied with the 
methods used to assess writing at our campus." Thus, many of the respondents are 
unhappy with their present circumstances in the area of writing assessment (See Table 
28). 

Table 28 

Respondent Opinions about the Assessment of Writing Skills by Size of Institution 

Small Medium Large Extra large 

Agree Disagree Agree Disagree Agree Disagree Agree Disagree 





# 


7r 


# 


% 


# 


7r 


# 


% 


# 


% 


# 


% 


# 


7c 


# 


7c 


Big Impact 


77 


79 


20 


21 


167 


72 


64 


28 


97 


78 


28 


22 


16 


67 


8 


33 


Good Influence 


90 


93 


7 


T 

/ 


193 


82 


42 


18 


103 


82 


22 


18 


22 


92 


2 


8 


Satisfied 


66 


68 


31 


32 


135 


56 


105 


44 


70 


55 


57 


45 


16 


67 


8 


33 
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SECTIO ill. RESPONSES Categorized by Public/Private Status of Institutk \ 



As noted in the x.itroduction to this report, based on data supplied by the United 
States Department of Education, the random sample drawn for the survey was stratified 
on the basis of the public or private status of the institution . A total of 280 public 
institutions and 254 private institutions responded to the survey. This section of the 
report provides results of responses to survey questions categorized by the 
public/private status of the institution. 



Assessment Methods and Decisions Based on Assessment Results 
Which Decisions are Based on the Assessmen t of Writing Skills? 

:he 534 respondents, the numbers and percentages of those who indicated 
whetht r not they based certain decisions, in whole or in part, on any type of 
assessment of writing skills categorized by the public/private status of the institution 
are as follows: 



Table 29 

Decisions Based on Any Type of Assessment of Writing Skills 
by Public/Private Status of Institution 



Admission 

Public Private 





# 


% 


# 


% 


Yes 


87 


33 


147 


62 


No 


178 


67 


92 


38 



Exemption 



Yes 
No 



Public Private 



# 


% 


# 


% 


200 


74 


183 


80 


72 


26 


45 


20 



Placement 



Yes 
No 



Public Private 



# 


% 


# 


% 


241 


89 


183 


78 


30 


11 


52 


22 



Proficiency 

Public Private 



Yes 
No 



# 


% 


# 


% 


186 


70 


118 


52 


78 


30 


111 


48 



Graduation 



Yes 
No 



Public Private 



# 


% 


# 


% 


33 


87 


88 


38 


176 


67 


144 


62 
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Following a similar pattern to that found for all respondents, for institutions 
which indicated that decisions on their campuses are based, in whole or in part, on any 
type of assessment of writing skills, most use some form of writing assessment for 
placement and exemption decisions. Seventy-four percent of the public institutions, 
and 80% of the private institutions report using some form of writing assessment for 
exemption purposes. Similarly, 89% of the public institutions, and 78% of the private 
institutions report using some form of writing assessment for placement purposes. 
Relatively fewer institutions use some form of writing assessment for the other 
assessment purposes. A considerable number, however, use writing assessment to 
make decisions regarding proficiency (70% of the public and 52% of the private 
institutions). A fairly large number of institutions report using writing assessment as a 
condition of graduation, 33% of the public and 38% of the private institutions. The 
results indicate that, overall, most of the decision-making based on writing assessment 
revolves around the freshman English course. 

What Methods are Used for the Different Assessment Purposes ? 

As noted in the section for all respondents, results in this section must be 
interpreted cautiously. Approximate numbers of institutions of public and private 
status using multiple choice exams, timed writing samples and/ or portfolios alone, or 
in combination with another method are reported in Table 30. The estimates for 
multiple choice and timed writing samples are based on the numbers of institutions 
who responded to specific questions regarding the development and implementation of 
those particular assessment methods. The estimates for portfoUos are based upon the 
number of institutions reporting their use for each of the purposes in response to 
question 2. Because respondents at a number of the institutions which use v.^riting 
assessment for particular decisions failed to respond to question 2, the numbers 
reported for portfoUos may be an underestimate of their actual use. 

Table 30 

Numbers of Assessments Using Multiple Choice Timed Writing Samples and/or 
Portfolios for Decision Making Reported by Public or Private Institutions 

Admission 

Public Private 





# 


7o 


# 


% 


Multiple Choice 


85 


98 


105 


71 


Timed Writing 


41 


47 


58 


39 


Portfolio 


4 


5 


29 


20 



Exemption 

Public Private 

# % # % 

Multiple Choice 1 37 69 111 61 

Timed Writing 136 68 1 23 67 

Portfolio 26 13 20 11 



Placement 





Public 


Private 




# 




# 




Multiple Choice 


171 


71 


120 


66 


Timed Writing 


172 


71 


140 


77 


Portfolio 


8 


3 


14 


8 


Proficiency 












Public 


Priv 


'ate 




# 




# 


7, 


Multiple Choice 


78 


42 


43 


36 


Timed Writing 


155 


83 


84 


71 


Portfolio 


66 


35 


45 


38 


Graduation 












Public 


Private 




# 


7r 


# 


'7c 


Multiple Choice 


45 


52 


31 


35 


Timed Writing 


70 


80 


55 


63 


Portfolio 


26 


30 


20 


23 



ERIC 



Responses indicate that of the institutions using multiple choice, timed writing 
samples and/ or portfolios to make decisions about admission, approximately 98% of 
the public institutions use multiple choice exams, compared to 71% of the private 
institutions. Approximately 47% of the public institutions use timed writing samples, 
compared to 39% of the private institutions. Approximately 5% of the public 
institutions use portfolios, compared to 20% of the private institutions. 

Of the institutions using one of the three methods to make decisions about 
exemption, approximately 69% of the public and 61% of the private institutions use 
multiple choice exams. Approximately 68% of the public institutions use timed writing 
samples, compared to 67% of the private. Approximately 13% of the public institutions 
use portfolios, compared to 11% of the private institutions. 

Of the institutions using one or more of the three methods to make decisions 
about placement, approximately 71% of the public and 66% of the private institutions 
use multiple choice exams. Approximately 71% of the public institutions use timed 
writing samples, compared to 77% of the private. Approximately 3% of the public 
institutions use portfolios, compared to 8% of the private institutions. 

Of the institutions using one or more of the three methods to make decisions 
about proficiency, approximately 42% of tl\e public and 36% of the private institutions 
use multiple choice exams. Approximately 83% of the public institutions use timed 
writing samples, compared to 71% of the private. Approximately 35% of the public 
institutions use portfolios, compared to 38% of the private institutions. 

Of the institutions using one or more of the three methods to make decisions 
about graduation, approximately 52% of the public and 35% of the private institutions 
use multiple choice exams. Approximately 80% of the public institutions use timed 
writing samples, compared to 63% of the private. Approximately 30% of the public 
institutions use portfolios, compared to 23% of the private institutions. 
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Practices Associated with Tinted Writing Samples 

As noted in the section for all respondents, several of the survey questions 
focused on the assessment process, asking v^ho develops, scores, and/ or makes 
decisions about the results of particular exams, hov^ particular exams are implemented 
and scored, and what kinds of feedback students receive. Tables in this section of the 
report provide data on these questions with responses categorized by the 
pubhc/private status of the institution. 

How Many Samples are Collected in Timed Writing Assessments ? 

Table 31 

Numbers of Writing Samples Collected in Writing Assessments 
Reported by Public and Private Institutions 

Admission 





Public 


Private 




# 


7c 


# 7c 


One 


16 


89 


30 79 


Two 


2 


11 


3 8 


Three 


0 


0 


{) 0 


Four or More 


0 


I) 


5 13 



Exemption 





Public 


Private 




# 


% 


# 7r 


One 


77 


78 


7b 80 


Two 


13 


13 


9 9 


Three 


5 


5 


4 4 


Four or More 


4 


4 


6 6 



Placement 

Public Private 





# 


% 


# 


% 


One 


104 


74 


101 


81 


Two 


10 


7 


14 


11 


Three 


5 


4 


1 


1 


Four or More 


22 


16 


9 


7 



Proficiency 

Public Private 





# 


% 


# 


7c 


One 


62 


56 


41 


61 


Two 


11 


10 


9 


13 


Three 


7 


6 


2 


3 


Four or More 


30 


27 


15 


22 



Graduation 




Public Private 





# 


% 


# 


% 


One 


37 


82 


29 


83 


Two 


2 


4 


4 


11 


Three 


2 


4 


1 


3 


Four or More 


4 


9 


1 


3 
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Of the institutions indicating that timed writing samples are collected to make 
decisions about admission, approximately 89% (16) of the public and 79% (30) of the 
private institutions rely on a single sample. For decisions about exemption, 
approximately 787o (77) of the public institutions which assess writing for this purpose 
rely on a single sample, compared to 80% (76) of the private. For decisions about 
placement, approximately 74% (104) of the public institutions rely on a single sample, 
compared to 81% (101) of the private. For decisions about proficiency, approximately 
567o (62) of the public institutions rely on a single sample, compared to 61% (41) of the 
private. For decisions about graduation, approximately 82% (37) of the public 
institutions which assess writing for this purpose rely on a single sample, compared to 
83% (29) of the private. 

How Much Time Is Allowed for Writing in Timed Writing Assessments ? 

As noted in the section for all respondents, the amount of time students are 
allowed to complete writing samples is an issue of concern to professionals who want to 
see a closer match between assessment procedures and teachers' views of effective 
pedagogy, in v/hich students are encouraged to plan and revise their writing. 
Assessments which allow brief amounts of time tap only first draft writing. To gather 
information about this issue, the survey asked respondents who used timed writing 
samples to make assessment decisions to indicate the amount of time allowed per 
sample. The results, categorized by the public/private status of the institution are 
presented in Table 32. 

Table 32 

Average Times Allowed to Complete a Writing Sample Reported by Public and Private Institutions 

Admission 





Public 


Private 




# 


% 


# % 


<30 


8 


32 


14 36 


31-60 


12 


48 


15 38 


61-90 


4 


16 


4 10 


90+ 


1 


4 


6 15 



Exemption 





Public 


Private 




# 


% 


# % 


<30 


12 


10 


19 18 


31-60 


52 


43 


49 47 


61-90 


25 


20 


20 19 


90+ 


33 


27 


16 15 



Placement 





Public 


Private 




# 




# % 


<30 


36 


20 


25 18 


31-60 


98 


55 


77 56 


61-90 


22 


12 


22 16 


90+ 


23 


13 


13 9 



46 



5;; 



Proficiency 





Public 


Private 




# 




# '7< 


<3() 


10 


7 


8 10 


31-60 


55 


39 


38 47 




30 


21 


15 18 


90+ 


46 


33 


21) 25 



Graduation 





Public 


Private 




# 




# 1c 


<30 


3 


5 


3 6 


31-60 


24 


40 


19 40 


61-90 


9 


15 


9 19 


90+ 


24 


40 


16 34 



Of the institutions indicating they use timed writing samples in assessment to 
make decisions about fldm/ssion, approximately 32% (8) of the public institutions allow 
30 minutes or less, compared to 36% (14) of the private institutions. Approximately 48% 
(12) of the public institutions allow 31 to 60 minutes, compared to 38% (15) of the 
private institutions. Approximately 16% (4) of the public institutions allow 61 to 90 
minutes per sample, compared to 10% (4) of the private. Approximately 4% (1) of the 
public institutions allow more than 90 minutes, compared to 15% (6) of the private 
institutions. 

Of the institutions indicating they use timed writing samples in assessment to 
make decisions d^oo\x\. exemption, approximately 10% (12) of the public institutions allow 
30 minutes or less, compared to 18% (19) of the private institutions. Approximately 437o 
(52) of the public institutions allow 31 to 60 minutes, compared to 47% (49) of the 
private institutions. Approximately 20% (25) of the public institutions allow 61 to 90 
minutes per sample, compared to 19% (20) of the private. Approximately 27% (33) of 
the public institutions allow more than 90 minutes, compared to 15% (16) of the private 
institutions. 

Of the institutions indicating they use timed writing samples in assessment to 
make decisions about placement, approximately 20% (36) of the public institutions allow 
30 minutes or less, compared to 18% (25) of the private institutions. Approximately 55% 
(98) of the public institutions allow 31 to 60 minutes, compared to 56% (77) of the 
private institutions. Approximately 12% (22) of the public institutions allow 61 to 90 
minutes per sample, compared to 16% (22) of the private. Approximately 13% (23) of 
the public institutions allow more than 90 minutes, compared to 9% (13) of the private 
institutions. 

Of the institutions indicating they use timed writing samples in assessment to 
make decisions about proficiency, approximately 7% (10) of the public institutions allow 
30 minutes or less, compared to 10% (8) of the private institutions. Approximately 39% 
(55) of the public institutions allow 31 to 60 minutes, compared to 47% (38) of the 
private institutions. Approximately 21% (30) of the public institutions allow 61 to 90 
minutes per sample, compared to 18% (15) of the private. Approximately 33% (46) of 
the public institutions allow more than 90 minutes, compared to 25% (20) of the private 
institutions. 



Of the institutions indicating they use timed writing samples in assessment to 
make decisions about graduation, approximately 5% (3) of the pubHc institutions allow 
30 minutes or less, compared to 6% (3) of the private institutions. Approximately 40% 
(24) of the public institutions allow 31 to 60 minutes, compared to 40% (19) of the 
private institutions. Approximately 15% (9) of the public institutions allow 61 to 90 
minutes per sample, compared to 19% (9) of the private. Approximately 40% (24) of the 
public institutions allow more than 90 minutes, compared to 347o (16) of the private 
institutions. 



Are Prewriting and /or Revising Activities Encouraged When Timed Writing Samples 
Are Collected? 



In order to gather information about the degree to which assessment procedures 
mirrored typical patterns in writing instruction, respondents were also asked to indicate 
whether or not they encourage prewriting and/or revising activities during timed 
writing tasks. Table 33 presents responses to these questions categorized by the 
public/private status of the institutions. 

Table 33 

Prewriting and/or Revising Activities in the Collection of Timed Writing Samples 
Reported by Public or Private Institutions 

Admission 

Public Private 

# 7o # % 

Yes 13 36 6 12 

No 23 64 43 «8 



Exemption 

Public Private 





# 


% 


# 


'7r 


Yes 


60 


47 


47 


42 


No 


69 


53 


65 


58 



Placement 

Public Private 





# 


% 


# 


% 


Yes 


97 


54 


72 


50 


No 


84 


46 


71 


50 



Proficiency 

Public Private 





# 


% 


# 


% 


Yes 


117 


76 


56 


62 


No 


36 


24 


34 


38 



Graduation 

PubHc Private 





# 


% 


# 


% 


Yes 


43 


62 


36 


65 


No 


26 


38 


19 


35 



48 '-^ ( 



Overall, the patterns of responses categorized according to the public or private 
status of the institution mirror the patterns found for all respondents together. That is, 
in most cases, there does not seem to be any important variation from the pattern 
according to the public/private status of the institution. The exceptions appear to be 
admission and proficiency. Of the institutions responding to this question, 36% (13) of 
the public institutions, but only 12% (6) of the private institutions indicated that they 
did encourage prewriting and/ or revising activities when timed writing samples were 
collected for admissions purposes. When timed samples were collected for exemption 
purposes, 47% (60) of the public institutions and 42% (47) of the private institutions 
indicated they encouraged these kinds of process activities. When timed samples were 
collected for placement purposes, 54% (97)' of the public institutions and 50% (72) of the 
private institutions indicated they encouraged these kinds of process activities. When 
timed samples were collected to assess proficiency, 76% (117). of the public institutions 
and 62% (56) of the private institutions indicated they encouraged these kinds of 
process activities. When timed samples were collected for graduation purposes, 62% 
(43) of the public institutions and 65% (36) of the private institutions indicated they 
encouraged prewriting and revising activities. 

Is Feedback Provided to Students about the Results of Timed Writing Assessments? 

As noted above, an issue in assessment procedures is the question of whether 
students should receive feedback about their performance. For pragmatic reasons, 
(expense, time) some institutions may not provide feedback. However, some educators 
argue that assessment should contribute to learning and that feedback is thus a 
necessary part of the assessment process. To gather information relevant to this issue, 
one question on the survey asked whether students are given feedback about their 
performance when timed writing samples were collected. Responses to this question 
are categorized in Table 34 according to the public/ private status of the institutions 
responding. 



Table 34 



Feedback to Students About Perfonnance on Timed Writing Sample(s) 
Reported by Public and Private Institutions 



Admission 



Public 



Private 



Yes, automatic 
Yes, if request 
No, no feedback 



# 

16 

9 

17 



% 
38 
21 
40 



# 
15 
15 
26 



% 
27 
27 
46 



Exemption 



Public 



Private 



Yes, automatic 
Yes, if request 
No, no feedback 



# 
52 

55 
33 



%. 
37 
39 
24 



# 
26 
72 
23 



% 
21 

a) 

19 



49^ 



Placement 



Public Private 





# 


% 


# 


'7f 


Yes, automatic 


78 


41 


35 


24 


Yl'S, if request 


81 


43 


83 


56 


N(i, no feedback 


30 


16 


30 


20 



Proficiency 

Public Private 





# 


7r 


# 


7r 


Yes, automatic 


79 


51 


41 


47 


Yes, if request 


69 


44 


37 


43 


No, no feedback 


8 


5 


9 


10 



Gra ''vaHon 

Public Private 





# 


7r 


# 


% 


Yes, automatic 


25 


37 


19 


35 


Yes, if request 


38 


56 


34 


63 


No, no feedback 


5 


7 


1 


2 



Of the institutions responding to this question, 40% (17) of the public and 46% 
(26) of the private institutions report that they do not provide feedback to students 
about their performance on timed writing samples during the admissions process. In 
the case of assessments conducted for exemption purposes, 24% (33) of the public and 
19% (23) of the private institutions report that they do not provide feedback. In the case 
of assessments conducted for placement purposes, 16% (30) of the public and 20% (30) 
of the private institutions report that they do not provide feedback. 

Participants in the Assessment Process 

As noted in the introduction, some professionals argue for the value of direct 
involvement of faculty in the assessment process. They point out that when faculty are 
involved in development assessments are more likely to be grounded in the faculty's 
"theories, curricula, and classroom practices" (Greenberg, 1992, 15). It has also been 
argued that participation in the assessment process can serve as a vehicle for effective 
staff development, with the caution that "a good test can have as positive an effect on 
mediocre teaching as a bad test has negative effects on teaching" (Lucas, 1988, 2). To 
gather information about the extent of faculty involvement in the writing assessment 
process in postsecondary institutions, specific questions about faculty participation 
were included in the survey. 

Who Develops Multiple Choice Exams ? 

Following the pattern for all respondents, among the institutions who use 
multiple choice exams, most exams are developed by the staff of a commercial agency 
(such as the ACT, SAT, or CLEP) for the assessment of writing at the pre-freshman and 
freshman levels for admission, regardless of the size of the institution. For admission, 
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88% of the public and 96% of the private institutions which responded to this question 
employ a commercial agency's exams. As noted previously, this finding is not 
surprising, since standardized tests are commonly used, particularly for admission and 
exemption (such as the AP English examination). Most institutions, regardless of their 
public/private status, report using a commercial exam for these purposes (see Table 35), 
In contrast to the pattern for admission assessments, and following the pattern for all 
respondents, the multiple choice exams used to assess proficiency are developed at 
most public institutions 63% (49) by their own faculty or staff, and at many private 
institutions by their own faculty or staff 47% (20), 

Table 35 

Participants in thz Development of Multiple Choice Exams by Public/Private Status of Institution 



Admission 



Exemption 



Placement 



Proficiency 



Graduation 





Public 


Private 




# 


% 


# 


7c 


Commercial Agency 


75 


88 


101 


96 


Faculty /Staff at College or University 


5 


6 


3 


3 


State Agency or Department 


5 


6 


1 


1 




Public 


Private 




# 


7r 


# 


% 


Commercial Agency 


102 


74 


96 


86 


Faculty /Staff at College or University 


25 


18 


14 


13 


State Agency or Department 


10 


7 


1 


1 




Public 


Pri\ 


'ate 




# 


7r 


# 


7< 


Commercial Agency 


126 


74 


97 


81 


Faculty /Staff at College or University 


26 


15 


19 


16 


State Agency or Department 


19 


11 


4 


3 




Public 


Private 




# 


% 


# 


7c 


Commei'cial Agency 


21 


27 


22 


51 


Faculty/Staff at College or University 


49 


63 


20 


47 


State Agency or Department 


8 


10 


1 


2 




Public 


Private 




# 


% 


# 


7c 


Commercial Agency 


12 


27 


12 


39 


Faculty /Staff at College or University 


20 


44 


16 


52 


State Agency or Department 


13 


29 


3 


10 



51 



Who Sets the Standards for Multiple Choice Exams? 



The individual who sets the standards (e.g., who decides which scores are 
acceptable) at institutions using multiple choice writing assessments for the purpose of 
admission is predominantly the administrator at the college or university, at both public 
(40, 49%) and private (68, 72%) institutions. Following the pattern for all respondents, a 
relatively large number of institutions report involving their faculty in decisions about 
admission (31, 38%) at public institutions and at private (25, 26%) institutions. As 
would be expected, however, faculty are typically not very active in the admissions 
process. In contrast to the pattern for decisions about admission, for all other purposes, 
however, the faculty at most institutions set the standards, regardless of the 
public/private status of the institution (see Table 36). 

Table 36 

Participants in Setting the Standards for Multiple Choice Exams by Public/Private Status of Institution 



Admission 



Pl«ia*ment 



Exemption 



Proficiency 





Public 


Private 




# 


7c 


# 


7r 


Administrator at Own Institution 


40 


49 


68 


72 


Faculty /Staff at Own Institution 


31 


38 


25 


26 


State Agency or Department 


11 


13 


2 


2 




Public 


Private 




# 


% 


# 


7c 


Administrator at Own Institution 


29 


21 


17 


17 


Faculty /Staff at Own Institution 


95 


69 


80 


81 


State Agency or Department 


13 


9 


2 


2 




Public 


Private 




# 


7f 


# 


7( 


Administrator at Own Institution 


20 


12 


15 


13 


Faculty /Staff at Own Institution 


130 


78 


104 


87 


State Agency or Department 


16 


10 


1 


1 




Public 


Private 




# 


% 


# 


7c 


Administrator at Own Institution 


4 


4 


5 


10 


Faculty/Staff at Own Institution 


81 


91 


47 


90 


State Agency or Department 


4 


4 


0 


0 



Graduation 



Administrator at Own Institution 
Faculty/Staff at Own Institution 
State Agency or Department 



Public 


Private 


# 7o 


# 7c 


5 10 


6 16 


32 67 


30 81 


11 23 


1 3 



52 



ERIC 



Who Develops the Topics for Timed Writing Samples? 



In contrast to the results for multiple choice exams, among the institutions that 
use timed writing samples for their assessment purposes, most of the assessments were 
deveh^ped by faculty or staff at their own colleges or universities. With regard to 
admission, most respondents indicated that their timed writing exams are developed by 
faculty or staff, 49% at public institutions and 55% at private, although many are 
developed by commercial agencies, 46% at pubUc institutions and 40% at private. With 
regard to exemption, most respondents indicated that their timed writing exams are 
developed by faculty or staff, regardless of the pubUc/private status of the institutions. 
Timed writing exams used for placement, proficiency, and graduation decisions are also 
predominantly developed by faculty or staff at their own colleges or universities 
regardless of the institution's status (see Table 37) . 

Table 37 

Participants in the Development of Timed Writing Exams by Public/Private Status of Institution 



Admission 





Public 


Private 




# % 


# '7^ 


Commercial Agency 


19 46 


23 40 


Faculty /Staff at College or Universitv 


20 49 


32 55 


State .Aeency or Department 


2 5 


3 5 


Exemption 








Public 


Private 




# % 


# 7r 


Commercial Agency 


42 31 


31 25 


Facult)' /Staff at College or University 


90 66 


90 73 


State Agency or Department 


4 3 


2 2 


Placement 








Public 


Private 




# 7r 


# % 


Commercial Agency 


12 7 


14 10 


Faculty /Staff at College or University 


151 88 


123 88 


State Agency or Department 


9 5 


3 2 


Proficiency 








Public 


Private 




# % 


# % 


Commercial Agency 


5 3 


3 4 


Faculty /Staff at College or University 


147 95 


81 96 


State Agency or Department 


3 2 


0 0 


Graduation 








Public 


Private 




# % 


# % 


Commercial Agency 


5 7 


4 7 


Faculty /Staff at College or University 


52 74 


49 8^ 


State Agency or Department 


13 19 


2 4 




Who Sets the Standards for the Evaluation " "imed Writing Samples? 

Data with regard to the individual \ .o sets the standards at institutions using 
timed writing exams reflected the same pattern as did responses in the previous 
question, as well as for the parallel question regarding setting standards for multiple- 
choice exams. For the admissions process, at many of the institutions, the decision 
comes from the "top down," or the administration, 38% at public institutions and 49% at 
private institutions. In most cases, however, standards for timed writing samples are 
set by faculty or staff (60% at public institutions compared to 51% at private institutions 
for the admissions process). For the remaining purposes, standards are set primarily by 
the faculty or staff, regardless of the public/private status of the institution, although 
standards for graduation are set by a state agency or department at fair number of 
public institutions (15, 21%). 

Table 38 

Participants in Setting the Standards for Timed Writing Exams by Public/Private Status of Institution 



Admission 



Administrator at Own Institution 
Faculty/Staff at Own Institution 
State Agency or Department 



Public 


Private 


# '7c 


# '7( 


15 38 


28 49 


25 60 


29 51 


2 5 


0 0 



Exemption 



Placement 



Froficiency 



Public 



Graduation 



Private 





# 


7c 


# 


9c 


Administrator at Own Institution 


n 


H 


9 


8 


Faculty/Staff at Own Institution 


122 


85 


110 


92 


State Agency or Department 


10 


7 


0 


0 




Public 


Pri\ 


'ate 




# 


% 


# 


7f 


Administrator at Own Institution 


• 4 


2 


6 


4 


Faculty /Staff at Own Institution 


160 


92 


139 


95 


State Agency or Department 


10 


8 


2 


1 




Public 


Private 




# 


% 


# 


% 


Administrator at Own Institution 


3 


2 


3 


3 


Faculty/Staff at Own Institution 


153 


97 


85 


97 


State Agency or Department 


2 


1 


0 


0 




Public 


Private 




# 


% 


# 




Administrator at Own Institution 


5 


7 


4 


7 


Faculty /Stiff at Own Institution 


52 


72 


51 


91 


State Agency or Department 


15 


21 


1 


2 




Who Scores the Timed Writing Samples? 



The individuals who score the timed writing samples reflect the pattern of 
practices reported in the data regarding test development and standard setting. For 
admission, nearly half of the scoring is conducted by a commercial agency at most 
instituKons, as would be expected if they are standardized tests, (48%) at public 
institutions, and (43%) at private institutions. For the other purposes, most timed 
writing sarxiples are scored by faculty or staff at their own institution, regardless of 
public,/private status (see Table 39). 

Table 39 

Scoring Practices for Timed Writing Samples by Public/Private Status of Institution 



Admission 



Exemption 



Placement 



Proficiency 



Graduation 





Public 


Private 




# 


% 


# 


7r 


Commercial Agency 


19 


48 


23 


43 


Faculty at Own Campus 


16 


40 


21 


39 


Testing Staff at Own Campus 


5 


13 


10 


19 




Public 


Private 




# 


7r 


# 


7r 


Commercial Agency 


30 


21 


21 


18 


Faculty at Own Campus 


105 


73 


89 


76 


Testing Staff at Own Campus 


8 


6 


7 


6 




' Public 


Private 




# 


% 


# 


7f 


Commercial Agency 


8 


5 


n 


8 


Faculty at Own Campus 


146 


87 


119 


88 


Testing Staff at Own Campus 


13 


8 


5 


4 




Public 


Private 




# 


% 


# 


% 


Commercial Agency 


1 


1 


1 


1 


Faculty at Own Campus 


153 


97 


80 


96 


Testing Staff at Own Campus 


4 


3 


2 


2 




Public 


Pri 


c^ate 




# 


% 


# 


% 


Commercial Agency 


5 


8 


5 


10 


Faculty at Own Campus 


52 


85 


42 


81 


Testing Staff at Own Campus 


4 


7 


5 


10 



Authority over the Assessment Process 

As noted in the introduction to this report, authority over the assessment process 
is an issue of concern to educators because externally mandated testing programs may 
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usurp faculty authority and diminish their professional prerogatives. To gather 
information relevant to this issue, the survey asked respondents to identify the primary 
governing authority over the assessment process for each assessment purpose. 



Table 40 

Governing Authority Over the Assessment Process by Public/Private Status of Institution 



Admission 





Public 


Private 




# 


7< 


TT /f 


State or Regional Authority 


55 


44 


O D 


Faculty Member or Group 


19 


15 


27 21 


ucan or L/tner Aaminisiraior 


32 


25 


90 70 


i^oiiege I rustees or universiry ixegenib 


20 


16 




Exemption 








Public 


Private 




# 


7c 


a <-7r 

ff /r 


State or Regional Authority 


14 


8 


1 


Faculty Member or Group 


127 


70 




Dean or Other Administrator 


33 


18 




College Trustees or University Regents 


7 


4 




Placement 










Public 


Private 




# 


7c 


f? k 


State or Regional Authority 


37 


16 


1 1 


Faculty Member or Group 


153 


67 




Dean or Other Administrator 


27 


12 




<^oiiege 1 rustees or universiiy Kegents 


12 


5 




I roiiLienLv 










Public 


Private 




# 


7r. 


# 7< 


State or Regional Authority 


6 


3 


0 0 


Faculty Member or Group 


153. 


87 


115 93 


Dean or Other Administrator 


13 


7 


7 6 


College Trustees or University Regents 


3 


2 


2 2 


Graduation 










Public 


Private 




# 


% 


# % 


State or Regional Authority 


21 


22 


4 5 


Faculty Member or Group 


49 


51 


61 77 


Dean or Other Administrator 


12 


12 


11 14 


College Trustees or University Regents 


15 


15 


3 4 



Respondents' reports indicate that 44% (55) of writing assessments at public 
institutions conducted for admission purposes are mandated by external authorities, but 
only 5% (6) at private institutions. Following the pattern for all respondents, faculty at 
public institutions appear to have more authority in assessments conducted for other 
purposes. Most assessments for exemption are mandated by a faculty member or group, 
70% (127) at public institutions and 827o (135) at private institutions. Most assessments for 



ERIC 



56 



6:: 



placement are mandated by a faculty member or group, 677o (153) at public institutions, 
and 86% (150) at private institutions, as are most assessments for proficiency, 87% (153) at 
public institutions, 93% (115) at private. Several assessments for placement, however, are 
mandated by a state or regional authority, 16% (37) at public institutions. 

Respondent Opinions 

As noted in the introduction, the three final questions on the survey elicited the 
opinions of respondents with respect to the assessment of writing skills at their 
institutions on a four-point scale ranging from "Agree Strongly," to "Disagree Strongly." 
Most respondents agreed or agreed strongly that the assessment of writing skills has had 
a big impact on writing instruction at their campuses. Further, most agreed or agreed 
strongly that writing assessment has had a good influence on writing instruction. Over 
half were satisfied with the writing assessment methods employed at their institutions. 
However, a considerable number of respondents for their institutions either disagreed or 
strongly disagreed with the statement "I am satisfied with the methods used to assess 
writing at our campus." Thus, many of the respondents are unhappy with their present 
circumstances in the area of writing assessment (See Table 41). 

Table 41 
Respondent Opinions 

A. The assessment of writing skills has had a big impact on writing instruction at our campus. 

Publi c Private 

Agree Strongly »3 32 41 19 

Agree ' 127 49 106 48 

Disagree 38 15 53 24 

Disagree Strongly 9 4 20 9 

B. The assessment of writing skills has had a good influence on writing instruction at our campus. 

Public Private 

# % # % 

Agree Strongly 87 33 38 17 

Agree 143 55 140 64 

Disagree 25 10 30 14 

Disagree Strongly 6 2 12 5 

C. I am satisfied with the methods used to assess writing at our campus. 

Public Private 

Agree Strongly 
Agree 
Disagree 

Disagree Strongly 

Following the pattern for all respondents, most of the respondents agreed, or 
agreed strongly, that the assessment of writing skills had a big impact and a good on 
writing instruction at their campuses. In a similar vein, most indicated satisfaction with 
the methods used to assess writing at their campuses (see Table 41). 



# 


% 


# 


% 


30 


12 


27 


12 


124 


48 


106 


46 


82 


32 


72 


31 


23 


9 


24 


10 



57 



SECTION IV* RESPONSES CATEGORIZED BY 2 YEAR/' 4 YEAR STATUS OF INSTITUTION 



As noted in the introduction to this report, based on data supplied by the United 
States Department of Education, demographic information about each institution was 
collected. In one question, respondents were asked to identify the 2 year or 4 year 
status of the institution. One hundred, seventy-four of the 2 year institutions and 340 of 
the 4 year institutions responded to this question. This section of the report provides 
results of responses to selected survey questions categorized by the reported 2 year/4 
year status of the institution. 



Assessment Methods and Decisions Based on Assessment Results 



Which Decisions are Based on the Assessment of Writing Skills ? 

The numbers and percentages of the respondents at 2 year and 4 year institutions 
who indicated whether or not they based certain decisions, in whole or in part, on any 
tvpe of assessment of writing skills are as follows: 

Table 42 

Decisions Based on Any Type of Assessment of Writing Skills 
Reported by 2 Year and 4 Year Institutions 



Admission 



2 Year 4 Year 



# % # 7r 

Yes 35 21 190 58 

No 130 79 1 39 42 



Exemption 

2 Year 4 Year 





# 


% 


# 


% 


Yes 


108 


65 


268 


82 


No 


58 


35 


57 


18 



Placement 

2 Year 4 Year 





# 




# 


7r 


Yes 


151 


91 


265 


SO 


No 


15 


9 


66 


20 



Proficiency 

2 Year 4 Year 





# 




# 




Yes 


lis 


74 


1S2 


56 


No 


42 


26 


143 


44 



Graduation 

2 Year 4 Year 





# 


1, 


# 




Yes 


3S 


24 


135 


41 


No 


122 


76 


191 





5« 



Following the pattern found for all respondents, for institutions which indicated 
that decisions on their campuses are based, in whole or in part, on any type of 
assessment of writing skills, most use some fonn of writing assessment for placement 
and exemption decisions and a substantial number use writing assessment for the 
purpose of assessing proficiency. Relatively fewer institutions use writing assessment 
for the other purposes. Between the two groups, however, the percentages sometimes 
vary widely. Not surprisingly, given the open admission policy of many 2 year 
institutions, relatively few of them reported assessing writing for admission purposes. 
Twenty-one percent of the 2 year institutions assessed writing for this purpose 
compared to 58% of the 4-year institutions. Sixty-five percent of the 2 year institutions 
report using some form of writing assessment for exemplion purposes. In comparison, 
82% of the 4 year institutions assess writing for this purpose. Thus, it seems that more 
of the 4 year institutions assess writing for the purposes of admission and exemption 
than do the 2 year institutions. 

For placement purposes, however, the pattern is reversed, with relatively more 
of the 2 year than the 4 year institutions reporting that they use writing assessment for 
placement purposes. Ninety-one percent of the 2 year institutions compared to 80% of 
the 4 year institutions report using some form of writing assessment for this purpose. A 
considerable number of institutions in both categories use writing assessment to make 
decisions regarding proficiency, but again, the percentages vary widely. Seventy-four 
of the 2 year institutions compared to 54% of the 4 year institutions report that they 
assess writing for the purpose of making decisions about proficiency. The results thus 
suggest that in comparison to 4 year institutions, relatively more of the 2 year 
institutions assess writing for placement and proficiency purpose. 

For making decisions about graduation, the pattern is reversed, with, not 
surprisingly, relatively more of the 4 year institutions assessing writing for this purpose. 
Twenty-four percent of the 2 year institutions report using writing assessment as a 
condition of graduation compared to 41% of the 4 year institutions. 

What Methods are Used for the Different Assessment Purposes ? 

As noted in the section for all respondents, results in this section must be 
interpreted cautiously. Approximate numbers of institutions of 2 year and 4 year status 
using multiple choice exams, timed writing samples and/ or portfoHos alone, or in 
combination with another method are reported in Table 43. The estimates for multiple 
choice and timed writing samples are based on the numbers of institutions who 
responded to specific questions regarding the development and implementation of 
those particular assessment methods. The estimates for portfolios are based upon the 
number of institutions reporting their use for each of the purposes in response to 
question 2. Because respondents at a number of the institutions which use writing 
assessment for particular decisions failed to respond to question 2, the numbers 
reported for portfolios may be an underestimate of their actual use. 




Table 43 

Approximate Numbers of Institutions Using Multiple Choice Timed Writing Samples and/or 
Portfolios for Decision Making Reported by 2 Year and 4 Year Institutions 



Admission 

2 Year 4 Year 





# 




# 


'7, 


Multiple Choice 


43 


99 


130 


79 


Timed Writing 


24 


56 


75 


39 


Portfolio 


2 


5 


29 


15 



Exemption 

2 Year 4 Year 





' # 




# 




Multiple Choice 


79 


73 


176 


66 


Timed Writing 


77 


71 


191 


71 


Portfolio 


U) 


9 


36 


13 



Placement 

2 Year 4 Year 





# 


% 


# 


7r 


Multiple Choice 


129 


85 


183 


69 


Timed Writing 


109 


72 


210 


79 


Portfolio 


3 


2 


19 


7 



Proficiency 

2 Year 4 Year 

# % # % 

Multiple Choice 61 52 65 36 

Timed Writing 102 86 136 75 

Portfolio 34 29 75 41 



Graduation 



Multiple Choice 
Timed Writing 
Portfolio 



2 Year 4 Year 



# 


% 


# 


% 


27 


71 


51 


38 


34 


89 


91 


67 


14 


37 


31 


23 
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Responses indicate that approximately 99% of the 2 year institutions which 
indicate they assess writing to make decisions about admission use multiple choice 
exams, compared to 60% of the 4 year institutions. Approximately 56% of the 2 year 
institutions use timed writing samples, compared to 39% of the 4 year institutions. 
Approximately 5% of the 2 year institutions use portfolios, compared to 15% of the 4 
year institutions. Of the institutions assessing writing to make decisions about 
exemption, approximately 73% of the 2 year and 66% of the 4 year institutions use 
multiple choice exams. Approximately 71% of the 2 year institutions use timed writing 
samples. The same percent (71%) of the 4 year institutions use timed writing samples. 
Approximately 9% of the 2 year institutions use portfolios, compared to 13% of the 4 
year institutions for making decisions about exemption. 

Of the institutions assessing writing to make decisions about placement, 
appioximately 85% of the 2 year and 69% of the 4 year institutions use multiple choice 
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exams. Approximately 72% of the 2 year institutions use timed writing samples, 
compared to 79% of the 4 year. Approximately 2% of the 2 year institutions report 
using portfolios to make placement decisions, compared to 7% of the 4 year institutions. 

Of the institutions assessing writing to make decisions about proficiency, 
approximately 52% of the 2 year and 36% of the 4 year institutions use multiple'choice 
exams. Approximately 86% of the 2 year institutions use timed writing samples, 
compared to 75% of the 4 year institutions. Approximately 29% of the 2 year 
institutions use portfolios to make decisions about proficiency, compared to 41% of the 
4 year institutions. 

Of the institutions assessing writing to make decisions about graduation, 
approximately 71% of the 2 year institutions use multiple choice exams, compared to 
38% of the 4 year institutions. Approximately 89% of the 2 year institutions use timed 
writing samples for this purpose, compared to 67% of the 4 year institutions. 
Approximately 29% of the 2 year institutions use portfolios, compared to 41% of the 4 
year institutions. 



Practices Associated with Timed Writing Samples 

As noted in the section for all respondents, several of the survey questions 
focused on the assessment process, asking who develops, scores, and /or makes 
decisions about the results of particular exams, how particular exams are implemented 
and scored, and what kinds of feedback students receive. Tables in this section of the 
report provide data on these questions with responses categorized by the 2 year/4 year 
status of the institution. 

How Many Samples are Collected in Timed Writing Assessments ? 

Table 44 

Numbers of Writing Samples Collected in Writing Assessments 
Reported by 2 Year and 4 Year Institutions 



Admission 










2 Year 


4 Year 




# 


% 


# % 


One 


9 


90 


34 83 


Two 


1 


10 


4 10 


Three 


0 


0 


0 0 


Four or More 


0 


0 


3 7 


Exemption 










2 


Year 


4 Year 




# 


% 


# % 


One 


46 


79 


107 80 


Two 


7 


12 


15 11 


Three 


3 


5 


3 2 


Four or more 


2 


3 


8 6 



hi 



Placoment 

2 Year 4 Year 





# 


'I 


# 




One 


63 


89 


139 


81 


Two 


4 


5 


20 


12 


Three 


2 


3 


4 


2 


Four or More 


2 


3 


8 


3 



Prc^ficiency 

2 Year 4 Year 





# 


7r 


# 


7r 


One 


45 


65 


58 


61 


Two 


6 


9 


12 


13 


Three 


5 


7 


4 


4 


Four or More 


13 


19 


21 


22 



Graduation 

2 Year 4 Year 





# 


7( 


# 


7c 


One 


14 


74 


52 


85 


Two 


1 


5 


5 


8 


Three 


0 


0 


3 


5 


Four or More 


4 


21 


1 


2 



Of the institutions indicating that timed writing samples are collected to make 
decisions about admission, approximately 90% (9) of the 2 year institutions and 
aoproximately 83?o (34) of the four year institutions rely on a smgle sample. Vox 
decisions about exemption, approximately 79% (46) of the 2 year institutions rely on a 
single sample, compared to 80% (107) of the 4 year. For decisions about placement 
approximately 89% (63) of the 2 year institutions rely on a single sample, compared to 
81% (139) of the 4 year. For decisions about proficiency, approximately 65% (45) of the Z 
year institutions rely on a single sample, compared to 61% (58) of the 4 year. For 
decisions about graduation, approximately 74% (14) of the 2 year institutions rely on a 
single sample, compared to 85% (52) of the 4 year. 

How Much Time Is Allowed for Writing in Timed Writ ing Assessments? 

As noted in the section for all respondents, the amount of time students are 
allowed to complete writing samples is an issue of concern to professionals who want to 
see a closer match between assessment procedures and teachers' views of effective 
pedagogy, in which students are encouraged to plan and revise their wnting^ 
Assessments which allow brief amounts of time tap only first draft wnting^ To gather 
information about this issue, the survey asked respondents who used timed writing 
samples to make assessment decisions to indicate the amount of time allowed per 
sample. The results, categorized by the 2 year/4 year status of the institution are 
presented in Table 45. 
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Table 45 

Average Times Allowed to Complete a Writing Sample Reported by 2 Year and 4 Year Institutions 



Admission 

2 Year 4 Year 





# 




# 


*7r 


<3() 


7 


44 


15 


33 


31-60 


6 


38 


19 


42 


61-90 


3 


19 


4 


9 


90+ 


0 


\) 


7 


16 



Exemption 

2 Year 4 Year 





# 


% 


# 


% 


<30 


6 


9 


24 


15 


31-60 


31 


45 


70 


45 


61-90 


18 


26 


27 


17 


90+ 


14 


20 


34 


22 



Placement 

2 Year 4 Year 





# 


% 


# 


% 


<30 


26 


25 


35 


17 


31-60 


53 


52 


119 


57 


61-90 


11 


11 


33 


16 


90+ 


12 


12 


23 


11 



Proficiency 

2 Year 4 Year 





# 




# 


7r 


<30 


8 


9 


10 


8 


31-60 


37 


40 


54 


43 


61-90 


21 


22 


23 


18 


90+ 


27 


29 


39 


31 



G raduation 

2 Year 4 Year 





# 


% 


# 


% 


<30 


3 


11 


3 


4 


31-60 


13 


46 


30 


38 


61-90 


4 


14 


14 


18 


90+ 


8 


29 


32 


40 



^Of the ir\stitutior\s indicating they use timed writing samples in assessment to 
make decisions about admission, approximately 44% (7) of the 2 year institutions allow 
30 minutes or less, compared to 33% (15) of the 4 year institutions. Approximately 38% 
(6) of the 2 year institutions allow 31 to 60 minutes, compared to 42% (19) of the 4 year 
institutions. Approximately 19% (3) of the 2 year institutions allow 61 to 90 minutes per 
sample, compared to 9% (4) of the 4 year. None of the 2 year institutions allow more 
than 90 minutes, compared to 16% (7) of the 4 year institutions. 

Of the institutions indicating they use timed writing samples in assessment to 
make decisions about exemption, approximately 9% (6) of the 2 year institutions allow 30 
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minutes or less, compared to 15% (24) of the 4 year institutions. Approximately 45% 
(31) of the 2 year institutions allow 31 to 60 minutes, compared to 45% (70) of the 4 year 
institutions. Approximately 26% (18) of the 2 year institutions allow 61 to 90 minutes 
per sample, compared to 17% (27) of the 4 year. Approximately 20% (14) of the 2 year 
institutions allow more than 90 minutes, compared to 22% (34) of the 4 year institutions. 

Of the institutions indicating they use timed writing samples in assessment to 
make decisions about placement, approximately 25% (26) of the 2 year institutions allow 
30 minutes or less, compared to 17% (35) of the 4 year institutions. Approximately 52%. 
(53) of the 2 year institutions allow 31 to 60 minutes, compared to 57% (119) of the 4 
year institutions. Approximately 11% (11) of the 2 year institutions allow 61 to 90 
minutes per sample, compared to 16% (33) of J:he 4 year. Approximately 12% (12) of 
the 2 year institutions allow more than 90 minutes, compared to 11% (23) of the 4 year 
institutions. 

Of the institutions indicating they use timed writing samples in assessment to 
make decisions about proficiency, approximately 9% (8) of the 2 year institutions allow 
30 minutes or less, compared to 8% (10) of the 4 year institutions. Approximately 40% 
(37) of the 2 year institutions allow 31 to 60 minutes, compared to 43% (54) of the 4 year 
institutions. Approximately 22% (21) of the 2 year institutions allow 61 to 90 minutes 
per sample, compared to 18% (23) of the 4 year. Approximately 29% (27) of the 2 year 
institutions allow more than 90 minutes, compared to 31% (39) of the 4 year institutions. 

Of the institutions indicating they use timed writing samples in assessment to 
make decisions about graduation, approximately 11% (3) of the 2 year institutions allow 
30 minutes or less, compared to 4% (3) of the 4 year institutions. Approximately 467o 
(13) of the 2 year institutions allow 31 to 60 minutes, compared to 38% (30) of the 4 year 
institutions. Approximately 14% (4) of the 2 year institution allow 61 to 90 minutes per 
sam.ple, compared to 18% (14) of the 4 year. Approximately 29% (8) of the 2 year 
institutions allow more than 90 minutes, compared to 40% (32) of the 4 year institutions. 

Are Prewriting and/or Revising Activities Encouraged When Timed Writing Samples 
Are Collected? 

In order to gather information about the degree to which assessment procedures 
mirrored typical patterns in writing instruction, respondents were also asked to indicate 
whether or not they encourage prewriting and/ or revising activities during timed 
writing tasks. Table 46 presents responses to these questions categorized by the 2 
year/4 year status of the institutions. 



Table 46 

Prewriting and/or Revising Activities in the Collection of Timed Writing Samples 
Reported by 2 Year and 4 Year Institutions 



Admission 



2 Year 



4 Year 



No 



# 

H 
15 



% 
35 
h5 



# 
11 
48 



7c 
19 
HI 
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Exemption 

2 Year 4 Year 





# 


7r 


# 


'7c 


Yes 


31 


43 


75 


45 


No 


41 


57 


92 


55 



Placement 

2 Year 4 Year 





# 




# 


'7, 


Yes 


50 


48 


117 


54 


No 


55 


52 


98 


46 



Proficiency 

2 Year 4 Year 





# 


'7, 


# 


'7r 


Yes 


69 


70 


102 


72 


No 


29 


30 


40 


28 



Graduation 

2 Year 4 Year 





# 


7r 


# 


7o 


Yes 


19 


54 


60 


68 


No 


16 


46 


28 


32 



Overall, the patterns of responses categorized by 2 year/4 year of institutions 
mirror the patterns found for all respondents together. That is, there does not seem to 
be any important variation from the pattern according to the 2 year/4 year status of the 
institution. The only exception may be practices during assessment for admission 
purposes. Of the institutions responding to this question, 35% (8) of the 2 year 
institutions, and 19% (11) of the 4 year institutions indicated that they did encourage 
prewriting and/or revising activities when timed writing samples were collected for 
admissions purposes. It should be noted however, that the percentages in this case are 
based on very small numbers. Only 24 of the 2 year institutions reported that they used 
timed writing assessment for admissions purposes, and of those, 23 responded to the 
question about feedback. 

When timed samples were collected for exemption purposes, 43% (31) of the 2 
year institutions and 45% (75) of the 4 year institutions indicated they encouraged these 
kinds of process activities. When timed samples were collected for placement purposes, 
50% (48) of the 2 year institutions and 54% (117) of the 4 year institutions indicated they 
encouraged these kinds of process activities. When timed samples were collected to 
assess proficiency, 70% (69) of the 2 year institutions and 72% (102) of the 4 year 
institutions indicated they encouraged these kinds of process activities. When timed 
samples were collected for graduation purposes, 54% (19) of the 2 year institutions and 
68% (60) of the 4 year institutions indicated they encouraged these kinds of process 
activities. 

Is Feedback Provided to Students about the Results of Timed Writing Assessments? 

As noted above, an issue in assessment procedures is the question of whether 
students should receive feedback about their performance. For pragmatic reasons. 
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(expense ne) some institutions may not provide feedback. However me educators 
argue tha ..-sessment should contribute to learning and that feedback .hus a 
necessary part of the assessment process. To gather information relevant to this issue, 
one question on the survey asked whether students had to receive feedback about their 
performance when timed writing samples were collected. Responses to this question 
are categorized in Table 47 according to the 2 year/4 year status of the institutions 
responding. 

Table 47 

Feedback to Students About Perfonnance on Timed Writing Sample(s) 



Reported by 2 Year and 4 Year Institutions 


Admission 










2 Year 


4 Year 




# 


7f 


# 7 


Yes, automatic 


10 


43 


19 27 


Yes, if request 


7 


31) 


17 24 


No, no feedback 


6 


26 


35 49 


Exemption 










2 Year 


4 Year 




# 




# 7c 


Yes, automatic 


35 


47 


43 23 


Yes, if request 


23 


31 


100 55 


No, no feedback 


16 


22 


40 22 


Placement 










2 Year 


4 Year 




# 


7r 


# 7c 


Yes, automatic 


52 


47 


61 27 


Yes, if request 


45 


41 


115 52 


No, no feedback 


14 


13 


46 21 


Proficiency 










2 Year 


4 Year 




# 


% 


# 7c 


Yes, automatic 


54 


55 


65 46 


Yes, if request 


40 


41 


64 45 


No, no feedback 


4 


4 


13 9 


Graduation 










2 Year 


4 Year 




# 


% 


# 7r 


Yes, automatic 


15 


47 


28 31 


Yes, if re<pest 


15 


47 


57 64 


No, no feedback 


2 


6 


4 4 



Of the institutions responding to this question, 26% (6) of the 2 year and 49% (35) 
of the 4 year insriturions report that they do not provide feedback to students about 
their performance on tinned writing samples during the admiss^ iis process. In the case 
of assessments conducted for exemption purposes, 22% (16) ot ae 2 year and 22% (40) 
of the 4 year institutions report that they do not provide feedback. In the case of 
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assessments conducted for placement purposes, 13% (14) of the 2 year and 21% (46) of 
the 4 year institutions report that they do not provide feedback. 



Participants in the Assessment Process 



As noted in the introduction, some professionals argue for the value of direct 
involvement of faculty in the assessment process. They point out that v^hen faculty are 
involved in development, assessments are more likely to be grounded in the faculty's 
"theories, curricula, and classroom practices" (Greenberg, 1992, 15). It has also been 
argued that participation in the assessment process can serve as a vehicle for effective 
staff development, with the caution that "a good test can have as positive an effect on 
mediocre teaching as a bad test has negative effects on teaching" (Lucas, 1988, 2). To 
gather information about the extent of faculty involvement in the writing assessment 
process in postsecondary institutions, specific questions about faculty involvement were 
included in the survey. 

Who Develops Multiple Choice Exams ? 

Following the pattern for all respondents, among the institutions who use 
multiple choice exams, most exams are developed by the staff of a commercial agency 
(such as the ACT, SAT, or CLE?) for the assessment of writing at the pre-freshman and 
freshman levels for admission, regardless of the size of the institution. For admission, 
74% of the 2 year and 97% of the 4 year institutions which responded to this question 
employ a commercial agency's exams. As noted previously, the widespread use of 
commercially developed exams is not surprising, since standardized tests are 
commonly used, particularly for admission and exemption purposes (such as the AP 
English examination). Most institutions responding to this question, regardless of their 
2 year/4 year status, report using a commercial exam for these purposes (see Table 48). 
However, following the pattern for all respondents, of those using multiple choice 
exams to assess writing proficiency for entry to the next level of coursework, the 
multiple choice exams are developed at most 2 year institutions 64% (38) by their own 
faculty or staff, and at many 4 year institutions by their own faculty or staff 47% (28). 
Surprisingly, approximately 48 percent of the 4 year institutions responding to this 
question reported that they used commercial exams for assessing writing proficiency. 



Participants in the Development of Multiple Choice Exams by 2 Year/4 Year Status of Institution 



Table 48 



Admisbion 



2 Year 



4 Year 



Commercial Agency 

Faculty /Staff at College or University 

SUte Agency or Department 



# 

29 

5 
5 



.7r 

74 

13 
13 



# 

144 

3 



% 
97 
2 



67 



Exemption 








2 Year 


4 Year 




# '7i 


# '■7( 


VwV.ll 1 11 1 Llnl /At'diLV 


51 67 


144 86 


Fnrnltv /Staff at C^(^lU^o^t-> or Univ'tTsitv 

1 {iv.uiiy / -J veil 1 <i I \ vii v.ll\> \ ' 1 w 1*1 vviw'tvv 


19 25 


19 11 


Statp AcTGncv or DeDartrnt?nt 


6 8 


5 3 


riacenient 








2 Year 


4 Year 




# 


# ^ 


iwOrnrntfrLidi r\^trnLY 


91 7h 


no 77 


Far 111 tv /Staff at Collect? or Univcrsitv 


15 13 


28 17 


Statp Ap'pnrv or Opnartmpnt 


13 11 


10 6 


Proficiencv 








2 Year 


4 Year 




tr fC 


# % 


VwOrnrn(rrcidi i^gcncy 


IS 25 


29 48 


Farnltv /Staff at (^nllpt^p or Uni vorsitv 


38 64 


28 47 


Statp Aapnrv f"ir Opnartmpnt 


6 10 


3 5 


Graduation 








2 Year 


4 Year 




# 7r 


# 7r 


Conrunercial Agency 


6 22 


18 37 


Faculty /Staff at College or University 


13 48 


23 47 


State Agency or Department 


8 30 


8 16 



Who Sets the Standards for Multiple Choice Exams? 

The individual who sets the standards (e.g., who decides which scores are 
acceptable) at institutions using multiple choice writing assessments for the purpose of 
admission is predominantly the administrator at the college or university, at 4 year (95, 
69%) institutions. In contrast, at 2 year institutions, the individual(s) who set the 
standard are primarily faculty or staff (21, 58%). Following the pattern for all 
respondents, a relatively large number of institutions report involving their faculty in 
assessment for exemption purposes, (51, 67%) at 2 year institutions and (121, 78%) at 4 
year institutions. For all other purposes it is predominantly the facult)' or staff at most 
institutions who set the standards, regardless of the 2 year/4 year status of the 
institution (see Table 49). 

Table 49 

Participants in Setting the Standards for Multiple Choice Exams by 2 Year/4 Year Status of Institution 

Admission 



Admmistrator at Own Institution 
Faculty /Staff at Own institution 
State Agency or Department 
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2 Year 


4 Year 


# 7r 


# % 


10 28 


95 69 


21 58 


34 25 


5 14 


8 6 







Exemption 












2 Year 


4 Year 




# 




# 


7r 


Administrator at Own Institution 


15 


20 


30 


19 


Faculty /Staff at Own Institution 


51 


67 


121 


78 


State Agency or Department 


10 


13 


5 


3 


Placement 












2 Year 


4 Year 




# 




# 


7r 


Administrator at Own Institution 


12 


10 


22 


13 


Faculty /Staff at Own Institution 


^1 


79 


139 


84 


State Agency or Department 


12 


10 


5 


3 


Proficiency 












2 Year 


4 Year 




# 


7r 


# 


7r 


Administrator at Own Institution 


1 


2 


7 ■ 


9 


Faculty /Staff at Own Institution 


57 


95 


68 


88 


State Agency or Department 


2 


3 


2 


3 


Graduation 












2 Year 


4 Year 




# 


7r 


# 


7r 


Administrator at Own Institution 


2 


8 


8 


14 


Faculty/Staff at Own Institution 


17 


65 


44 


77 


State Agency or Department 


7 


27 


5 


9 



Who Develops the Topics for Tim^d Writing Samples? 

In contrast to the results for multiple choice exams, among the institutions that 
use timed writing samples for their assessment purposes, most of the assessments were 
developed by faculty or staff at 2 year institutions. Of the institutions responding to this 
question, 74% (17) of the 2 year institutions indicate that faculty or staff develop their 
timed writing exams for admission purposes. However, proportionately fewer of the 4 
year institutions have exams for this purpose which are developed by faculty or staff 
(33, 467o). Timed writing exams are developed by commercial agencies at 49% of the 
institutions responding to this question. These data suggest that 4 year institutions may 
rely more often on the commonly used standardized tests which include a writing 
sample than 2 year institutions. 

With regard to exemption, most respondents indicated that their timed writing 
exams are developed by faculty or staff, regardless of the 2 year/4 year status of the 
institutions. Timed writing exams used for placement, entry, and graduation decisions 
are also predominantly developed by faculty or staff at their own colleges or 
universities regardless of the institutions status, although a fair number of 2 year 
institutions report using timed writing exams developed by a state agency or 
department (10, 29%) (see Table 50), 
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Table 50 

Participants in the Development of Timed Writing Exams by 2 Year/4 Year Status of Institution 



Admission 



Exemption 



Placement 



Proficiency 



Graduation 



Commercial Agency 

Faculty /Staff at College or University 

State Agency or Department 



Commercial Agency 

Faculty /Staff at College or University 

State Agency or Department 



Commercial Agency 

Faculty /Staff at College or University 

State Agency or Department 



Commercial Agency 

Faculty /Staff at College or University 

State Agency or Department 



Commercial Agency 

Faculty /Staff at College or University 

State Agency or Department 



2 Year 


4 Year 


# 7r 


# 7r 


5 22 


35 49 


17 74 


33 46 


1 4 


3 4 


2 Year 


4 Year 


# 7r 


# 7r 


17 22 


54 30 


55 72 


123 69 


4 5 


2 1 


2 Year 


4 Year 


# 7r 


# 7r 


9 8 


16 8 


92 87 


178 89 


5 5 


7 3 


2 Year 


4 Year 


# 'It 


# 7 


4 4 


4 3 


95 93 


130 97 


3 3 


0 0 


2 Year 


4 Year 


# % 


# 7r 


3 9 


6 7 


21 62 


79 88 


10 29 


5 5 



W ho Sets the Standards for the Evaluation of Timed Writing Samples? 

Data with regard to the individual who sets the standards at institutions using 
timed writing exams reflected the same pattern as did responses in the previous 
question, a pattern which diverges somewhat from the pattern found for all 
respondents. For the admissions process, at most of the 2 year institutions responding 
to this question, the decision does not come from the "top down," or the administration 
Rather, in many cases, standard setting is done by faculty and/or staf^ (16, 73%). At 4 
year institutions, in contrast, the responsibility for standard setting is equally divided 
between administrators (36, 49%) and faculty/staff (36, 49%). For the remaining 
purposes, standards are set predominantly by the faculty or staff, regardless of the 2 
year/4 year status of the institution (see Table 51), although timed writing exams at a 
fair number of 2 year institutions have standards set by a state agency or department 
(11,32%). 

70 




Table 51 

Participants in Setting the Standards for Timed Writing Exams by 2 Year/4 Year Status of Institution 



Admission 



Exemption 



Placement 



Proficiency 



Graduation 





2 Year 


4 Year 




# 




# 




Administrator at Own Institution 


5 


23 


36 


49 


Faculty/Staff at Own Institution 


16 


73 


36 


49 


State Agency or Department 


1 


4 


1 


1 




2 Year 


4 Year 




# 


7f 


# 


7r 


Administrator at Own Institution 


7 


9 


13 


7 


Faculty/Staff at Own Institution 


61 


80 


167 


92 


State Agency or Department 


8 


11 


1 


1 




2 Year 


4 Year 




# 


% 


# 


7r 


Administrator at Own Institution 


3 


3 


7 


3 


Faculty/Staff at Own Institution 


92 


89 


203 


95 


State Agency or Department 


8 


8 


4 


2 




2 Year 


4 Year 




# 


7r 


# 


7r 


Administrator at Own Institution 


2 


2 


4 


3 


Faculty /Staff at Own Institution 


97 


96 


138 


97 


State Agency or Department 


2 


2 


0 


0 




2 Year 


4Y 


ear 




# 


% 


# 


7r 


Administrator at Own Institution 


1 


3 


8 


9 


Faculty /Staff at Own Institution 


22 




80 


86 


State Agency or Department 


11 


32 


5 


5 



Who Scores the Timed Writing Samples? 

The individuals who score the timed writing samples reflect tlie pattern of 
practices reported in the data regarding test development and standard setting. For 
admission, nearly half of the scoring is conducted by a commercial agency at most 4 
year institutions (51%), as would be expected if they are standardized tests. At 2 year 
institutions, however, (13, 59% ) of the institutions responding to this question report 
that exams are scored by faculty at their ovm campus. For the other purposes, most 
tinned writing samples are scored by faculty or staff at their own institution, regardless 
of 2 year/4 year status (see Toble 52). 
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Table 52 

Scoring Practices for Timed Writing Samples by 2 Year/4 Year Status of Institution 



Admission 



Exemption 



Placement 



Proficiency 



Graduation 





2 Year 


4 Year 




# 




# 




C()mmercial Agencv 


5 


23 


35 


51 


Faculty at Own Campus 


13 


59 


22 


32 


Tf'stincy Sf;iff at Own Camnus 


4 


18 


11 


16 




2 Year 


4 Year 




# 




# 


7r 


Commercial Agencv 


12 


16 


38 


21 


Faculty at Own Campus 


56 


75 


134 


75 


1 L.ollllii Dvnll « l V-/VVII V^<11IIL/U3 


7 


9 


8 


4 




2 Year 


4 Year 




# 


7r 


# 


7c 


Commercial Agency 


4 


4 


15 


8 


Faculty at Own Campus 


84 


85 


179 


90 


Tocfinrr Qfraff :af Own I ;^mmi*? 


11 


11 


5 


2 




2Y 


ear 


4 Y 


ear 




# 


7r 


if 


7f 


Commercial Agency 


1 


1 


1 


1 


Faculty at Own Campus 


96 


95 


134 


98 


Testing Staff at Own Campus 


4 


4 


2 


1 




2 Year 


4 Year 




# 


7c 


# 


7r 


Commercial Agency 


4 


13 


6 


7 


Faculty at Own Oimpus 


23 


77 


70 


85 


Testing Staff at Own Campus 


3 


10 


6 


7 



Authority over the Assessment Process 

As noted in the introduction to this report, authority over the assessnnent process 
is an issue of concern to educators because externally mandated testing programs may 
usurp faculty authority and diminish their professional prerogatives. To gather 
information relevant to this issue, the survey asked respondents to identify the primary 
governing authority over the assessment process for each assessment purpose. 
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Table 53 

Governing Authority Over the Assessment Process by 2 Year/4 Year Status of Institution 



Admibsion 



Exemption 



riacoment 



Proficiency 



Graduation 



2 Year 



4 Year 





# 


f( 


# 




State or Regional Authority/ 


42 


58 


19 


11 


Facultv Member or Group 


10 


14 


34 


19 


Dean or Othc^r Administrator 


12 


16 


108 


61 


College Trustees or University Regents 




12 


16 


9 




2 Year 


4 Year 




# 


7< 


# 


7r 


State or Regional Authorit\' 


11 


11 


4 


2 


Faculty Member or Group 


60 


59 


199 


83 


Dean or Other Administrator 


26 


26 


34 


14 


College Trustees or University Regents 


4 


4 


3 


1 




2Y 


ear 


4 Y 


ear 




# 


7r 


# 




State or Regional Authority 


27 


18 


11 


4 


Faculty Member or Group 


88 


60 


211 


83 


Dean or Other Administrator 


23 


16 


25 


10 


College Trustees or University Regents 


8 


5 


6 


2 




2 Year 


4 Year 




# 




# 


7 


State or Regional Authority 


5 


4 


1 


1 


Faculty Member or Group 


97 


87 


168 


91 


Dean or Other Administrator 


8 


7 


12 


-r 

/ 


College Trustees or University Regents 


2 


2 


3 


2 




2 Year 


4 Year 




# 


7r 


# 


% 


State or Regional Authority 


12 


26 


13 


10 


Faculty Member or Group 


21 


46 


88 


69 


Dean or Other Administrator 


8 


17 


15 


12 


College Trustees or University Regents 


5 


11 


12 


9 



Respondents' reports indicate that 58% (42) of writing assessments at 2 year 
institutions conducted for admission purposes are mandated by external authorities, 
but only 11% (19 ) at 4 year institutions. Following the pattern for all respondents, 
faculty appear to have more authority in assessments conducted for other purposes, 
although the proportions vary somewhat according to 2 year or 4 year classification. 
Most assessments for exemption are mandated by a faculty member or group, 59% (69) 
.it 2 year inshtutions, but proportionately more 83% (199) at 4 year institutions. Most 
assessments for placement are mandated by a faculty member or group, 60% (88) at 2 
year institutions, but proportionately more 83% (211) at 4 year institutions. Authority 
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for making decisions about proficiency appears to be distributed similarly, regardless of 
the 2 year/4 year status of the institution. Of the 2 year institutions responding to this 
question, 87% (97) report faculty as the governing authority, 91% (168) at 4 year 
institutions. Several assessments, hov^ever, are mandated by a state or regional 
authority at 2 year institutions, for placement (27, 18%)) and for graduation (12, 26% ). 

Respondent Opinions 

As noted in the introduction, the three final questions on the survey elicited the 
opinions of respondents with respect to the assessment of writing skills at their 
institutions on a four-point scale ranging from "Agree Strongly," to "Disagree Strongly." 
Most respondents agreed or agreed strongly that the assessment of writing skills has 
had a big impact on writing instruction at their campuses, regardless of their 2 year/ 4 
year classification. Proportionately more of the respondents at 2 year institutions (62, 
39%), however, agreed "strongly" than did respondents at 4 year institutions (62, 20%). 
Approximately equal proportions of the respondents at 2 year institutions (81, 50) and 
at 4 year (152, 49%) simply agreed. On the whole, then, proportionately more of the 
respondents at 2 year institutions agreed that the assessment of writing skills has had a 
big impact on writing instruction at their campuses, 89% at 2 year institutions and 69% 
at 4 year institutions. A similar pattern appears with respect to the statement "The 
assessment of writing skills has had a good impact on writing instruction at our 
campus. Proportionately fewer respondents at 4 year institutions (1)6, 18%) strongly 
f-^.greed with the statement in comparison to respondents at 2 year institutions (69, 42%). 
Over half of the institutions were satisfied with the writing assessment methods 
employed at their institutions, regardless of their 2 year or 4 year classification, and the 
proportions of those who agreed strongly and agreed are similar at each type of 
institution. Following the pattern for all respondents, a considerable number of 
respondents either disagreed or strongly disagreed with the statement "I am satisfied 
with the methods used to assess writing at our campus," (62, 38%) at 2 year institutions 
and (138, 44%) at 4 year institutions. Thus, many of the respondents are unhappy with 
their present circumstances in the area of writing assessment at both 2 year and 4 year 
institutions (See Table 54). 
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Table 54 

Respondent Opinions About the Assessment of Writing Skills by 2 Year/4 Year Status of Institution 



The absesbment of writing skills has had a bi^ impact on writing instruction at our campus. 

2 Year 4 Year 

# # 

Agree Strongly 62 39 62 20 

Agree ' 81 50 152 49 

Disagree 15 9 75 24 

Disagree Strongly 3 2 23 7 • 



The assessment of writing skills has had a j^ood impact on writing instruction at our campus. 



Agree Strongly 

Agree 

Disagree 

Disagree Strongly 



2 Year 


4 Year 


# 7< 


# 7< 


69 42 


56 18 


82 49 


200 64 


11 7 


44 14 


4 2 


13 4 



I am satisfied with the methods used to assess writing at our campus. 

2 Year 4 Year 

Agree Strongly 24" 14 33 10 

Agree ' 80 48 149 47 

Disagree 49 30 104 33 

Disagree Strongly 13 8 34 11 
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SECTION V. RESPONDENTS' REPORTS OF SATISFACTION OR DISSATISFACTION 
RELATED TO SELECTED SURVEY QUESTIONS 



Additional analyses compared respondents' reports of satisfaction and 
dissatisfaction with assessment methods with their responses to other questions. Ot 
interest is the cross-tabulation of data reported about assessments in which 
particular methods are employed with respondents' reports of satisfaction or 
dissatisfaction. Table 55 shows the results for each assessment purpose and for all 
assessment purposes combined. Of the total number of assessments in which 
multiple choice methods were employed (for all purposes combined) 59% of these 
respondents indicated satisfaction with the methods employed. Of the assessments 
in which timed writing samples were used, 62% of these respondents indicated 
satisfaction. Of the assessments in which portfolios were used, 69% of these 
respondents indicated satisfaction. Thus, when all purposes are combined, the 
proportions of respondents who are satisfied when multiple choice and timed 
writing methods are employed at their campuses appear to be similar, regardless of 
the particular method. In relation to the other two methods, slightly more 
respondents report satisfaction when portfolios are used. 

Table 55 

Numbers of Respondent Reports of Satisfaction or Dissatisfaction with Assessments Using Multiple 
Choice, Timed Writing Samples and/or Portfolios for Decision Making 

All Purposes Combined 





Satisfied 


Dissat 


isfied 




# 


% 


# 


% 


Multiple Choice 


520 


59 


363 


41 


Timed Writing 


614 


62 


369 


38 


Portfolio 


163 


69 


74 


31 


Admission 












Satis 


fied 


Dissatisfied 




# 




# 


% 


Multiple Choice 


101 


55 


81 


45 


Timed Writing 


55 


60 


37 


40 


.■""^rtfolio 


19 


68 


9 


32 


Exemption 












Satisfied 


Dissatisfied 




# 


% 


# 


% 


Multiple Choice 


133 


56 


103 


44 


Timed Writing 


150 


61 


97 


39 


Portfolio 


31 


70 


13 


30 


Placement 












Satii 


fied 


Dissatisfied 




# 


% 


# 


% 


Multiple Choice 


162 


59 


114 


41 


Timed Writing 


184 


62 


112 


38 


Portfolio 


13 


65 


7 


35 



7A 



Proficiency 





Satis 


fied 


Diss< 


^tisfied 




# 


7r 


# 


% 


Multiple Choice 


77 


66 


39 


34 


Timed Wriring 


149 


65 


79 


35 


Portfolio 


69 


68 


33 


32 



Graduation 

Satisfied Dissatisfied 





# 


'7, 


# 


7r 


Multiple Choice 


47 


64 


26 


36 


Timed Writing 


76 


63 


44 


37 


Portfolio 


31 


72 


12 


2H 



Table 56 shows the results of the cross-tabulation of respondents' reports of 
satisfaction and dissatisfaction with the reported numbers of samples collected in 
timed writing exams. Of the reports of assessments in which one sample was 
collected (all purposes combined), 63% of the respondents indicated satisfaction; 66% 
when two samples were collected; 56% when three were collected; and 65% when 
four or more were collected. Thus, the proportions of respondents who report 
satisfaction appear to be roughly similar, regardless of the number of samples 
collected. 

Table 56 

Numbers of Respondent Reports of Satisfaction or Dissatisfaction 
Related to the Numbers of Writing Samples Collected in Writing Assessments 





Satisfied 


Dissatisfied 




# 




# 


One 


353 


63 


207 37 


Two 


46 


66 


24 34 


Three 


14 


56 


11 44 


Four or More 


60 


65 


32 35 


Admissioi, 










Satisfied 


Dissatisfied 




# 


7r 


# 7r 


One 


22 


50 


22 50 


Two 


3 


60 


2 40 


Three 


0 


0 


0 0 


Four or More 


3 


60 


2 40 


Exemption 


Satisfied 


Dissatisfied 




# 


% 


# 7r 


One 


96 


65 


52 35 


Two 


16 


76 


5 24 


Three 


3 


38 


5 62 


Four or More 


7 


70 


3 30 



Placement 





Satib 


fied 


Dissatisfied 




# 


'7r 


# 


% 


One 


127 


64 


73 


36 


Two 


13 


62 


8 


38 


Three 


2 


33 


4 


67 


Four or More 


15 


50 


15 


50 



Proficiency 





Satib 


fied 


Dissatisfied 




# 


'7i 


# 




One 


66 


64 


37 


36 


Two 


10 


59 


7 


41 


Three 


7 


78 


2 


22 


Four or More 


31 


74 


11 


26 



Graduation 





Satii 


5fied 


Dissatisfied 




# 


7c 


# 


7r 


One 


42 


65 


23 


35 


Two 


4 


67 


2 


33 


Three 


2 


KX) 


0 


0 


Four or More 


4 


80 


1 


20 



A similar pattern was seen in the cross-tabulation of respondents' reports of 
satisfaction or dissatisfaction with the reported time allowed in timed writing exams 
(see Table 57). Of the reports of assessments in which 30 minutes or less are allowed, 
(all purposes 'combined), 64% of the respondents also reported satisfaction. Of the 
reports of assessments in which 31 to 60 minutes are allowed, 61% of the 
respondents also reported satisfaction. Of the reports of assessments in which 61 to 
90 are allowed, 62% of the respondents also reported satisfaction. Of the reports of 
assessments in which more than 90 minutes are allowed, 68% of the respondents 
also reported satisfaction. Thus, the proportions of respondents who report they are 
satisfied appear to be roughly similar, regardless of the time allowed for writing. 

Table 57 

Numbers of Respondent Reports of Satisfaction or Dissatisfaction 
Related to the Average Times Allowed to Complete a Writing Sample 

All I^jrposes Combined 





Satisfied 


Dissatisfied 




# 


% 


# 




<30 


86 


64 


49 


36 


31-60 


258 


61 


164 


39 


61-90 


95 


62 


58 


38 


40+ 


125 


68 


58 


32 




78 



















Admission 



Satisfie d Dissatisfied 

# 7r # '-/r 

<3() 12 55 10 45 

31-60 13 50 13 50 

61-90 5 71 2 29 

90+ 3 50 3 50 



Exemption 



Satisfied Dissatisfied 
# V. # 7c 

<30 17 59 12 41 

31-60 57 58 41 • 42 

61-90 30 70 13 30 

90+ 33 75 11 25 



Placement 



Satisfied Dissatisfied 

# '7r # % 

<30 39 65 21 35 

31-60 99 59 68 41 

61-90 24 55 20 45 

90+ 22 71 9 29 



Proficiency 



Satisfied Dissatisfied 
# 7, # 7f 

<30 14 78 4 22 

31-60 62 • 70 26 30 

61-90 23 55 19 45 

90+ 43 65 23 35 



Graduation 



<30 
31-60 
61-90 

90+ 



Satisfied 



# 

4 

27 

13 

24 



% 
67 
63 
76 
67 



Dissatisfied 



# 
2 

16 
4 
12 



7r 

33 
37 
24 
33 



A somewhat different pattern begins to appear, however, in relation to pre- 
writing and revising activities and in reported consequences for students. Cross- 
tabulations of respondents reports of satisfaction and dissatisfaction with responses 
to questions about these aspects of the assessments suggest that proportionally fewer 
respondents are satisfied with the assessment methods employed at their campuses 
when prewriting and revising activities are not encouraged (all purposes combined). 
Sixty-nine percent of the respondents indicated they were satisfied with the 
assessment methods employed at their campuses when the assessments were those 
in which prewriting and revising activities were encouraged. A smaller number 
(56%) of the respondents reported satisfaction when the assessments were those in 
which prewriting and revising activities were not encouraged (see Table 58/. 



Table 58 

Numbers of Respondent Reports of Satisfaction or Dissatisfaction Related to the 
Prewriting and/or Revising Activities in the Collection of Timed Writing Samples 

All Purposes Combined 

Satisfied Dissatisfied 





# 


'I 


# 


f7 

'< 


Yes 


357 


69 


163 


31 


No 


249 


56 


198 


44 



Admission 

Satisfied Dissatisfied 





# 


7c 


# 


7. 


Yes 


13 


68 


6 


32 


No 


33 


54 


28 


46 



Exemption 

Satisfied Dissatisfied 





# 


'I 


# 


'7i 


Yes 


72 


70 


31 


30 


No 


69 


54 


58 


• 46 



Placement 

Satisfied Dissatisfied 





# 


% 


# 


7r 


Yes 


104 


65 


56 


35 


No 


86 


57 


64 


43 



Proficiency 

Satisfied Dissatisfied 





# 


7 


# 


7 


Yes 


114 


70 


49 


30 


No 


37 


55 


30 


45 



Graduation 

Satisfied Dissatisfied 





# 


7r 


# 


7 


Yes 


54 


72 


21 


28 


No 


24 


57 


18 


43 



A similar pattern occurs with regard to feedback (all purposes combined). For 
most of the assessments in which feedback is provided (71% automatic and 61% if 
requested) respondents reported satisfaction. Fewer (50%) of the respondents 
reported satisfaction with the methods employed at their campuses when feedback 
to students is not provided (see Table 59). 
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Table 59 

Numbers of Respondent Reports of Satisfaction or Dissatisfaction Related to 
Feedback to Students About Performance on Timed Writing Sample(s) 



Feedback to Students About Performance on Timed Writing SampIeCs/ 





Satisf 


ied 


Dissati 


sfied 




# 


'/f 


ff 


7i 


Yes, automatic 


263 


71 


110 


29 


Yes, if request 


286 


61 


185 


39 


No, no feedback 


H5 


50 


85 


50 


Admission 












Satii 


)fied 


Dissa 


tisfied 




# 


7r 


# 


7( 


Yes, automatic 


22 


71 


9 


29 


Yes, if request 


11 


46 


13 


54 


No, no feedback 


21 


54 


18 


46 


Exemption 












Sati; 


sfied 


Dissa 


tisfied 




# 


7c 


# 


7r 


Yes, automatic 


53 


69 


24 


31 


Yes, if request 


79 


66 


41 


34 


No, no feedback 


23 


45 


28 


55 


Placement 












Sati 


sfied 


Dissa 


tisfied 




# 


% 


# 


7c 


Yes, automatic 


75 


70 


32 


30 


Yes, if request 


94 


60 


63 


40 


No, no feedback 


29 


50 


29 


50 


Proficiency 












Sati 


sfied 


Dissa 


tisfied 




# 


'7c 


# 


7c 


Yes, automatic 


81 


70 


34 


30 


Yes, if request 


60 


60 


41^ 


40 


No, no feedback 


10 


59 


7 


41 


Graduation 












Sati 


sfied 


Dissa 


tisfied 




# 


7c 


# 


% 


Yes, automatic 


32 


74 


11 


26 


Yes, if request 


42 


60 


28 


40 


No, no feedback 


2 


40 


3 


60 



A similar pattern occurs in relation to consequences for students who fail to 
qualify for the regular freshman English course. When all purposes are combined, 
fewer respondents report satisfaction when alternatives indicated in the survey 
(courses, tutoring, retaking the exam), are unavailable. Sixty-four percent of the 
respondents report satisfactiX)n when an alternative is required, 58% when an 
alternative is optional, and relatively fev/er (51%) when not an option. 



ERLC 



Table 60 

Numbers of Respondent Reports or Satisfaction or Dissatisfaction Related to the Consequences for 
Students Who Fail to Qualify for the Regular Freshman English Course 



All Con^quences Combined 





Satisfied 


Dissatisfied 




# 




# '7, 


Yes, required at this campus 


339 


64 


189 3h 


Yes, ()pn(.)nai ai inis Ldmpuj 


144 


58 


1(1S 42 


ino, not cin upnun di inij Lcinipua 


166 


51 


157 49 


T^ike develonmpntal course for credit. 










Satisfied 


Dissatisfied 




# 


7c 


# 7f 


Yes, required at this campus 


94 


64 


53 36 


1 es, upiu)n<ti di uiio Ldiiipuj 


15 


56 


12 44 


No, not an option at this campus 


54 


55 














Satisfied 


Dissatisfied 




# 


7c 


# 7c 


Yes, required at this campus 


136 


62 


82 38 


Yes, optional at this campus 


17 


53 


^R 17 

1 0 *r/ 


No, not an option at this campus 


27 


47 




Attend tutoring. 










Satisfied 


Dissatisfied 




# 


7c 


# 7 


Yes, required at this campus 


59 


63 


34 37 


Yes, optional at this campus 


89 


60 


59 40 


No, not an option at this ci^mpus 


20 


51 


19 49 


Retake exam until performance is satisfactory. 








Satisfied 


Dissatisfied 




# 


7c 


# 7c 


Yes, required at this campus 


50 


71 


20 29 


Yes, optional at this campus 


23 


55 


19 45 


No, not an option at this campus 


65 


51 


63 49 



In regard to consequences for students who fail assessment of proficiency, a 
similar pattern appears. When all consequences are combined, sixty-six percent of 
the respondents report satisfaction when one of the alternative consequences (take 
designated writing course, prepare a portfolio, retake the exam) is required, 71% 
when it is optional, and relatively fewer (55%) when not an option (see Table 61). 

Table 61 

Numbers of Respondent Reports of Satisfaction or Dissatisfaction Related to the Consequences for 
Students Who Fail Assessment of Proficiency for Entry into a Next Level of Coursework 

All Consequences Combined 

Satisfied Dibsatibfied 
# 7 # 7 

Yes, required at this campus 202 102 34 

Yes, optional at this campus ^7 71 40 29 

No, nut an option at this campus 236 55 194 45 

82 



Prepare a portfolio and appeal. 

Satisfied Dissatisfied 

# 7r # 7r 



Yes, required at this campus 


5 


83 


1 


17 


Yes, optional at this campus 


40 


70 


17 


30 


No, not an option at this campus 


76 


56 


59 


44 


Take and oass a designated vvritine course. 










Satisfied 


Dissatisfied 




# 


7r 


# 


Iv 


Yes, required at this campus 


134 


65 


73 


35 


Yes, optional at this campus 


22 


. 69 


10 


31 


No, not an option at this campus 


18 


44 


23 


56 


Retake exam an unlimited number of times. 










Sati 


sflcd 


Dissatisfied 




# 


7r 


# 


% 


Yes, required at this campus 


33 


69 


15 


31 


Yes, optional at this campus 


18 


69 


8 


31 


No, not an option at this campus 


74 


57 


56 


43 


Retake exam a limited number of times 












Satisfied 


Dissatisfied 




# 


7r 


# 


7r 


Yes, required at this campus 


30 


70 


13 


30 


Yes, optional at this campus 


17 


77 


5 


33 


No, not an option at this campus 


68 


55 


56 


45 



A somewhat similar pattern appears in regard to consequences for students 
who fail assessment as a condition for graduation. Seventy percent of the 
respondents report satisfaction when one of the alternative consequences (i-ake a 
designated writing course, prepare a portfolio, retake the exam) is required, 56% 
when it is optional, and 59% when not an option (Table 62). 

Table 62 

Numbers of Respondent Reports of Satisfaction or Dissatisfaction Related to the 
Consequences for Students Who Fail Assessment as a Condition for Graduation 

A ll Consequences Combined 

Satisfied Dissatisfied 
# % # % 

Yes, required at this campus 93 70 40 30 

Yes, optional at this campus 48 56 38 44 

No, not an option at this campus 170 59 118 41 

Pre pare a_ portfolio and appeal. 

Yes, required at this campus 
' Yes, optional at this campus 

No, not an option at this campus 



Satisfied 


Dissatisfied 


# 7r 


# % 


3 75 


1 25 


14 67 


7 33 


56 62 


35 38 




Take and pass a designated writing course. 

Satisfied 
# 'I 
48 73 
17 50 
29 56 



Yes. required at this campus 
Yes, optional at tliis campus 
No, not an option at this campus 



Retake exam an unlimited number of times 



Dissatisfied 
# 'I 
18 27 
17 50 
23 44 





Satis 


fied 


Dissatisfied 




# 




# 




Yes, required at this campus 


28 


65 


15 


35 


Yes, optional at this campus 


11 


55 


9 


45 


No, not an option at this campus 


42 


60 


28 


40 


Retake exam a limited number of times. 












Satii 


)fied 


Dissatisfied 




# 


9J 


# 


7r 


Yes, required at this campus 


14 


70 


6 


30 


Yes, optional at this campus 


6 


55 


5 


45 


No, not an option at this campus 


43 


57 


32 


43 



The data suggest then, that respondents may be somewhat less likely to be 
satisfied with the assessment methods employed at their campus when the 
assessments do not incorporate prewriting and revising activities and when 
students do not have recourse to alternative paths for satisfying requirements when 
they fail an assessment. Nevertheless, the numbers of respondents indicating 
satisfaction with the methods employed at their campuses is somewhat surprising. 

Table 63 

Numbers of Respondent Reports of Satisfaction or Dissatisfaction Related to the 
Governing Authority for tlie Assessment Process 



All Purposes Combined 



State or Regional Authority 
Faculty Member or Group 
Dean or Other Administrator 
College/ University Trustees or Regents 



Satisfied 



# 
86 
577 
129 
41 



61 
61 
49 
61 



Dissatisfied 



# 
56 
363 
132 
26 



7r 

39 
39 
51 
39 



Admission 



State or Regional Authority 
Faculty Member or Group 
Dean or Other Administrator 
College/ University Trustees or Regents 



Satisfied 



# 

30 
28 
58 
13 



% 
51 
62 
50 
52 



Dissatisfied 



# 

29 

17 

57 

12 



% 
49 
38 
50 
48 
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0. 



Exemption 





Satisfied 


Dissatisfied 




# 


7c 


# % 


State or Regional Authority 


8 


53 


7 47 


Faculty Member or Group 


155 


62 


95 38 


Dean or Other Administrator 


27 


47 


31 53 


College/ University Trustees or Regents 


5 


71 


■ 2 29 


Placement 










Satisfied 


Dissatisfied 




# 


% 


# % 


State or Regional Authority 


27 


73 


10 27 


Faculty Member or Group 


168 


58 


121 42 


Dean or Other Administrator 


24 


51 


23 49 


College/ Universit\' Trustees or Reeents 


9 


69 


4 31 


Proficiencv 










Satisfied 

U Gf 


Dissatisfied 


State or Regional Authority 


n 

5 


83 


# % 
1 17 


Faculty Member or Group 


161 


63 


93 37 


Dean or Other Administrator 


9 


47 


10 53 


College/ University Trustees or Regents 


3 


60 


2 40 


Graduation 










Satisfied 


Dissatisfied 




# 


% 


# 7c 


State or Regional Authority 


16 


64 


9 36 


Faculty Member or Group 


65 


64 


37 36 


Dean or Other Administrator 


11 


50 


11 50 


College/University Trustees or Regents 


11 


65 


6 35 



The results for all purposes combined suggest that proportionately fev^ 
respondents report satisfaction when assessments are mandated by a dean or other 
administrator than when they are mandated by other groups. When all purposes are 
combined, 61% of the respondents report satisfaction when the assessments are 
mandated by a state or regional authority, 61% when a faculty member or group is 
the governing authority, 49% when the assessments are mandated by a dean or 
other administrator, and 61% when the assessments are mandated by college 
trustees or university regents (see Table 63). 



Conclusion 

The portrait of writing assessment practices in the United States provided by 
the survey results is not always encouraging. Given the generally common use of 
standardized tests such as the ACT and the SAT for admission and exemption 
purposes, it is not surprising that the use of multiple choice methods for assessing 
writing should be reported so frequently. However, given expressed concern among 
many professionals about the limitations of multiple-choice tests, in particular the 
mismatch of such tests with emerging theoretical conceptions of writing as a 
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complex, multi-faceted activity, the rather widespread use of multiple choice exams 
to assess writing proficiency for entry into the next level of coursework and the use 
of multiple choice methods to assess writing as a condition for graduation (457o of 
the 172 institutions that assess writing for this purpose) is dismaying. It should be 
noted, however, tha^ the estimates for use of multiple choice exams reported in 
Table 3 include institutions which are using multiple choice exams in combination 
with another method. Of the institutions which report using the assessment of 
writing to make decisions about admission, 82 (35%) indicated that they relied on 
multiple choice methods alone, about exemption, 42 (11%), about placement 92 
(22%), but about proficiency, only 6 (2%) and about graduation, only 7 (4%). 

The results also suggest that a growing number of institutions have turned to 
direct assessment of writing, particularly for making decisions about exemption, 
placement, proficiency and graduation purposes. For each of these purposes, over 
707c of the institutions which assess writing report using timed writing samples. 
The results also suggest a fairly widespread use of direct assessment of writing for 
admission purposes.- Forty-four percent of the institutions v/hich assess writing to 
make decisions about admission use timed writing samples for this purpose. Given 
the growing challenge to the practice of single sample assessment of writing, 
however, the finding that so many institutions rely on single sample assessment i^ a 
matter of concern. Across all purposes, very few institutions collect more than one 
sample when timed writing samples are collected. Again, however, it should be 
noted that many institutions may be collecting single samples in combination with 
some other method such as multiple choice assessment, portfolios, or a method not 
addressed by the survey. Of the institutions which assess writing for each of the 
purposes identified by the survey, relatively few indicated that they relied solely on 
timed writing assessment for decisions about admission 18 (8%), about exemption 76 
(207o), about placement 107 (187o). In relation to the other purposes, the practice of 
relying solely on the collection of timed writing samples to make decisions about 
proficiency 88 (267o) and about graduation 57 (337o) was somewhat more frequent. 
However, one would also expect that information from any particular assessment 
method or combination of methods would be supplemented by information from 
other sources such performance in coursework. Nevertheless, the widespread use 
of single sample assessment of writing is surprising. 

The number of respondents who report that they are satisfied with the 
assessment methods employed at their campuses is also somewhat surprising, 
particularly in relation to multiple choice methods. Again, however, it should be 
noted that the multiple choice methods may have been employed in combination 
with other methods, a not uncommon practice. Also, since the survey question 
asked respondents about satisfaction with methods as a whole, regardless of 
purpose, the survey could not address specific areas of concern that respondents may 
have had. Further, the satisfaction reported by many respondents may also have 
been influenced generally by a recognition that assessment can provide useful 
information to faculty and administrators alike and have a positive impact on 
curriculum. In general, most respondents (757o) agreed that the assessment of 
writing skills had a good influence on writing instruction at their campuses. 
Nevertheless, a substantial number of respondents disagreed with this positive 
view of the impact of assessment. Their disagreement reminds us that while 
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assessments of writing may have potential benefits, they may be based on 
inadequate methods, they may interfere with the educational process, and they may 
misrepresent the abilities of some students. 

Heartening though, is the indication of widespread faculty involvement at 
those institutions where writing is assessed for the various purposes. The data 
suggest that faculty are very involved in several stages of the assessment process for 
each of the purposes, especially so, subsequent to admission. Further, there seems to 
be no widespread trend toward externally mandated assessment. For the assessment 
purposes of granting exemption, placing students in appropriate courses and 
determining proficiency for entry into next levels of coursework and for graduation, 
in most cases, members of the faculty govern the assessment process. 
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Participants in Setting the Standards for Timed Writing Exams by 2 Year/4 Year 

Status of Institution 




2 year Proficiency 



4 year Proficiency 



2 year Graduation 



4 year Graduation 
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Participants in Setting the Standards for Multiple Choice Exams by 
2 Year/4 Year Status of Institution 



H Administrator at Own ^ Faculty/Staff at Own fl State Agency or 
Institution Institution Department 




BEST COPY AVAILABLE 



Participants in Setting the Standards for Timed Writing Exanis by 2 Year/4 Year 

Status of Institution 



■ Adntinistrator at Own ^ Faculty/Staff at Own ■ State Agency or Department 

Institution Institution 




2 year 4 year 2 year 4 year 2 year 4 year 

Admission Admission Exemption Exemption Placement Placement 



Scoring Practices for Timed Writing Samples (Who Scores Timed Writing Samples?) 




Governing Authority Over the Assessment Process (By Whom Mandated?) 




Respondent Opinions About the Assessment of Writing Skills (N-534) 



■ The assessment of writing □ The assessment of writing ■ I am satisfied with the 

skills has had a big impact on skills has had a good methods used to assess 

writing instruction at oiu* influence on writing writing at our campus, 

campus. instruction at our campus. 



60 - 



50 




% AgreeStnmgfy % Agree % Disagree % Disagree Stion^y 



BEST COPY AVAILABLE 



Numbers of Respondent Reports of Satisfaction or Oissatisfaction with Assessments 
Using Multiple Choice, Timed Writing Samples and/or Portfolios for Decision 

Kwdng 




](),. 



Numbers of Respondent Reports of Satisfection or Dissatisfaction Related to the 
Numbers of Writing Samfdes Collected In Writing Assessments 



70 T 




■ % Satisfied 
S % Dissatisfled 



One Two Three Four or More 



Numbers of Respondent Repents of Satisfaction or E)i58atisfaction Related to the 
Average Times Allowed to Complete a Writing Sample 




Numbers of Respondent Reports of Satisfaction or Dissatisfaction Related to 
the Prewriting and/or Revising Activities in the Coiiection of Timed Writing 

Samples 



70 -r 




% Satisfied % Dissatisfied 



^'K BEST COPY AVAILABLE 

ERIC 



Numbers of Respondent Repports of Satisfaction or Dissatishiction Related to 
Feedback to Students About Performance on Timed Writing Sample(s) 




Approximate Numbers of Assessments Using Multiple Choice Timed Writing 
Samples and /or Portfolios for Decision Making Reported by 2 Year and 4 Year 

Institutions 



j ■ Multiple Choice S Timed Writing ■ Portfolio 

I 




Approximate Numbers of Assessments Using Multiple Choice Timed Writing 
Samples and /or Portfolios for Decision Making Reported by 2 Year and 4 Year 

Institutions 




Participants in the Assessment Process (Who Sets the Standards for 

Multiple Choice) 



■ Administrator at own w Faculty/ staff at own Staff at state agency or 
college or university college or university department 



100 -r 

i 




Adnussion Exemption Placement Proficiency Graduation 



< i: 
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Participants in the Assessment Process (Who Sets the Standards for 
Writing Samples) 




Feedback to Students About Performance on Hmed Writing Samples 
(Opportunity for Students to Receive Feedback) 



I % Yes^ automatic ^ % Yes^ if requested H % No^ no feedback 
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Kinds of Feedback Provided To Students About Timed Writing Samples 



1 Students Receive Scores 


H Students Receive Written 


SS Students Receive Comments 




Comments 


in Conference 




Scoring Methods for Timed Writing Samples (Numbers of 
Institutions Using Single Scoring Methods) 



Analytic 



M General Impression ■ Primary Trait 



60 — 



50 



40 ^ 



30 



20 -H 



1 0 



0 -H 




Admission Exemption Placement Proficiency Graduation 




! h; 



BEST COPY AVAILABLE 



Scoring Methods for Timed Writing Sannples (Niunbers of Institutions Using 
Conrtbinations of Scoring Methods) 




Average Times Allowed to Complete a Writing Sample 




Prewriting and/or Revising Activities in the Collection of Timed Writing 

Samples 




Decisions Based on Any Type of Assessment of Writing Skills (N=534) 



80 - 




Admission Exemption Placement Proficiency Graduation 



ERIC 



BEST COPY AVAILABLE 



Numbers of Writing Samples Collected 




Scoring Practices for Timed Writing Samples by 2 Year/4 Year Status of 

Institution 



■ Commercial Agency ^ Faculty at Own Campus B Testing Staff at Own 

Campus 




2 year Proficienq' 4 year Proficiency 2 year Graduation 4 year Graduation 



Approximate Numbers of Institutions Using Multiple Choice, Timed Writing 
Samples and /or Portfolios for Decision Making 




Numbers of Respondents Reports of Satisfoction or Dissatisfaction Related to the 
Consequences for Students Who Fail Assesment of ProBdency for Entry into a 

Next Level of Coursework 
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Numbers of Respondent Reports of Satisfaction or Dissatisfaction Related to 
d« Consequences for Students Who Fail Assessment as a Condition ^ 

Graduation 




■ % Satisfied 
S % Dissatisfied 



Yes, required at tfus Yes, optional at this No, not an option at 
campus campus this campus 



Participants in Setting the Standards for Multiple Choice Exams by 2 Year/4 

Year Status of Institution 




Scoring Practices for Timed Writing Samples by 2 Year/4 Year Status of 

Institution 



H Commercial Agency 


M Faculty at Own Campus ■ Testing Staff at Own 




Campus 



90 -y 

1 




2 year 4 year 2 year 4 year 2 year 4 year 

Admission Admission Exemption Exemption Placen^ent Placement 



127 



Consequaices for Students (For Proficiency for Entry into a Next 

Level) 



% Required at This Campus G % Optional at This Campus 



% Not an Option at This 
Campus 



80 -T- 




Prepare a Portfolio and Take/pass Designated Retake Exam Retake Exam Limited « 
Appeal English Course Unlimited Times Until of Times 

Wittiout Credit Pass 



4 



Numbers d Respondent Reports of Satisibction or Dissatisfiaction Related to the 
Conaequences for Students Who Fail to Qualify for Ae Regular Freshman English 

Course 



70 -r 



60 — 



50 — 



40 -- 



30 — 



20 — 



10 ~ 




% Satisfied 
% Dissatisfied 



Yes, required at this 
campus 



Yes, optional at ttiis No, not an option at tfiis 
can^nis campus 



1 ') 
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Consequences for Students (For Graduation) 



% Required at This Campus □% Optional at This CamptJS 



% Not an Option at This 
Campus 



80 -r 



70 — 



60 -- 



50 



40 



30 



20 



10 — 




Prepare a Portfolio and Take/pass Designated 
Appeal Eng^ Course 

Without Credit 



Retake Exam 
Unlimited Untet Until 
Pass 



Retake Exam limited « 
of Times 



1 ;]t; 



ERIC 



BEST COPY AVAILABLE 



Consequences for Students (For Placement) 




Take/pass Take/pass 
Developmental English Developmental En^ish 
Course for Qedit Course Without Credit 



Attend Tutoring 



